Why Single Page Application

Single Page Applications (SPA) are built on expanding reach via the browser, reducing round tripping, and enhancing User Experience (UX). In this blog, we will discuss why we need SPA, its advantages and what different components required.

Why Single Page Application

Why do we want to write single page apps? The main reason is that they allow us to offer a more-native-app-like experience to the user.

This is hard to do with other approaches. Supporting rich interactions with multiple components on a page means that those components have many more intermediate states (e.g. menu open, menu item X selected, menu item Y selected, menu item clicked). Server-side rendering is hard to implement for all the intermediate states – small view states do not map well to URLs.

Single page apps are distinguished by their ability to redraw any part of the UI without requiring a server round-trip to retrieve HTML. This is achieved by separating the data from the presentation of data by having a model layer that handles data and a view layer that reads from the models.

How Loading Time Affects Your Bottom Line

  • A survey revealed that Loading time is a major contributing factor to page abandonment.
  • Page Load time is greater than 3 seconds, 20% of visitors will abandon the page.
  • Walmart found that every 100ms reduction in loading time leads to an increase in incremental revenue by as much as 1%.
  • Mozilla saw increase in downloads of Firefox by 5 million per month, after making their page 2.2 seconds quicker

Let’s understand Single Page Application

Single Page App - 1

Single Page Application is a web app in which the majority of interactions are handled on the client without the need to reach a server, with a goal of providing a more fluid user experience

Single Page Application Architecture

Single page apps typically have

  • “Application like” interaction
  • Back/Forward Button works as expected
  • More JavaScript than actual HTML
  • Ability to go Offline
  • Dynamic data loading from the server-side API, Works with Restful Web Services
  • Fluid transitions between page states, Richer Interaction between UI Components
  • Markup, CSS & JS required for the application sent with the initial request
  • Page changes occur via JavaScript using Templates & DOM manipulation
  • URL hashes are used to track state and provide bookmark-able links

Single Page App - 3

SPA Core Benefit

  • Performance Improvement
  • SPAs can improve performance in several ways: – Load time: 1 file each of HTML, CSS, JS
  • Static files not dynamic
  • Less data transfer: XHR calls only send – raw data, not HTML markup
  • Load distribution: dramatically less load on your server, by distributing it to clients
  • Caching gets a lot easier and more beneficial

SPA Benefits

  • It is an application so it can do Cool Stuff!
  • Responsive, native application feel. No page flicker!
  • Faster UI, More Interactive
  • Most of the page processing happens client side, offloading server load
  • Massive tooling support, Vibrant community
  • Code reuse. REST endpoints are general purpose and can be used for multiple applications
  • UI is just another Client
  • Perfect for supporting multiple platforms

Business Usage of SPA

  • Create “App-like user experience”
  • Build Hybrid (native) HTML5 applications
  • Mobile version of your web site The SPA sweet spot is likely not on web sites, but on content-rich cross-platform mobile apps
  • All data has to be available via some sort of API – this is a big advantage in our case as we want to have an API to our application anyway. Right now about 60-70% of my calls to get/update data are done through a REST API. Doing a single page application will allow us to better test our REST API since the application itself will use it. It also means that as the application grows, the API itself will grow since that is what the application uses; no need to maintain the API as an add-on to the application.
  • More responsive application – since all data loaded after the initial page is kept to a minimum and transmitted in a compact format (like JSON), data requests should generally be faster, and the server will do slightly less processing.

AngularJS for SPA implementation

AngularJs Logo

Key Features of AngularJS

  • Declarative HTML approach
  • Easy Data Binding : Two way Data Binding
  • Reusable Components
  • MVC/MVVM Design Pattern
  • Dependency Injection
  • End to end Integration Testing / Unit Testing
  • Routing
  • Templating

HTML Compiler

  • Angular’s HTML compiler allows the developer to teach the browser new HTML syntax. The compiler allows you to attach behaviour to any HTML element or attribute and even create new HTML elements or attributes with custom behaviour. Angular calls these behaviour extensions directives.
  • Compiler is an angular service which traverses the DOM looking for attributes. The compilation process happens in two phases. **Compile: traverse the DOM and collect all of the directives. The result is a linking function. ** Link: combine the directives with a scope and produce a live view. Any changes in the scope model are reflected in the view, and any user interactions with the view are reflected in the scope model. This makes the scope model the single source of truth.

AngularJS Way

Angularjs Arch

Grunt, The JavaScript Task Runner

Grunt The JavaScript Task Runner

Grunt is a task-based command line build tool for JavaScript projects.

Why use a task runner?

In one word: automation. The less work you have to do when performing repetitive tasks like minification, compilation, unit testing, linting, etc, the easier your job becomes. After you’ve configured it, a task runner can do most of that mundane work for you—and your team—with basically zero effort.

Why use Grunt?

The Grunt ecosystem is huge and it’s growing every day. With literally hundreds of plugins to choose from, you can use Grunt to automate just about anything with a minimum of effort. If someone hasn’t already built what you need, authoring and publishing your own Grunt plugin to npm is a breeze.

Bower, A package manager for the web

Bower, A package manager for the web

Bower is a package manager for the web. It offers a generic, unopinionated solution to the problem of front-end package management, while exposing the package dependency model via an API that can be consumed by a more opinionated build stack. There are no system wide dependencies, no dependencies are shared between different apps, and the dependency tree is flat.

Bower runs over Git, and is package-agnostic. A packaged component can be made up of any type of asset, and use any type of transport (e.g., AMD, CommonJS, etc.).

Live Implementation

proversity_logo

https://www.proversity.com is real implementation of Single Page Application, please go to website and browse through it. You can see some lag while opening first page, but after that all pages will open on blink of an eye, yes that can take some to get data from server which is very small chunk because we are just retrieving delta from server.

by Barkat Dhillon

ASPState database – A performance bottleneck

The Project

We were recently tasked with resolving a performance bottleneck on a very popular online food ordering system in India. The problem statement was rather simple : The website was running very slow during peak hours.

The application was opaque to us – being an off the shelf application whose IP and therefore source the application vendor retained. So our investigations and analysis would need to be limited to the OS, Hardware and application level configuration.

Every good technician brings his own preferred set of tools with him – New Relic is our tool of choice for monitoring. It comes with a free 30 day evaluation with some strings attached. This suited our purpose fine. [Note: It turned out later that the customer had a global enterprise licensing agreement for new relic!]

We looked at the metrics provided by New relic and could determine that the database was performing well. All the stored procedures were completing their operations in under 100 ms. Network latency & bandwidth between the application and the database was also found to be of no concern.

This was a three tier application and the web tier was indeed taking upto 15 secs to process requests. We decided to work down the tiers to identify and isolate the bottlenecks.

The load balancer was configured to use sticky sessions however the session was being stored in the ASPState database. Given that the LB configuration was sticky we decided to change the session configuration to InProc. And Voila the problems were all gone.

We had a really happy customer – who saw increase in revenue, reduction in provisioned hardware (therefore costs) and a 10 fold decrease in response time. Simple, right?

The above does point to the fact the ASPState database was a bottleneck and data also demonstrated that it was bottlenecking on Disk I/O. Why would that be?

Anytime a web request was coming in there was an write occurring to the ASPState database. Though the project/client did not need any of the below but here are a few pointers on how to tune the ASPState database in case your architecture requires its use. Please note that there is a third option called in SessionServer where Session State can be stored on a separate server in memory. Both SessionState and ASPState lead to a single point of failure and bottleneck.

In another .Net Application that we are building ground up we have decided to go stateless as this leads to way more scalable architectures.

Recommendations on tuning ASPState database

    1. Choose Simple Recovery Model - This will dramatically reduce the impact to the log that can be caused by the high volume of (transient and largely disposable) writes.  Read more

ALTER DATABASE ASPState SET RECOVERY SIMPLE;

2. Minimize or Isolate I/O  - Choose the sstype -c and -d options to our the ASPState database on a different disk.

3. Though possible never use the same ASPState database for two different web applications or sites.

4. The stored procedure tempresettimeout is called on every request and this can be madness on a high volume/many pageview per visit site. I am not sure  entirely sure how to address this but probably we can run this on some requests and not all?

5. There is a deleteexpiredtokens stored procedure that kills the performance under load. Run it sporadically and if possible not under peak load.

Bottomline

Coming from the world of Java/J2EE and Play Framework I dislike single points of failure and session state. In the .Net world an apparently harmless configuration can lead to ominous bottlenecks unless identified early.

My recommendation remains to build stateless applications and pass cookie information on each request rather than store state server side. This way it also becomes much easier to add new application server nodes to the cluster if required.

Zencoder HLS Encoding with Encryption

How to Deliver Awesome Looking HLS Video to Desktop and Mobile

Video Content providers understand the importance of a multi-device strategy, but earlier there was no single protocol compatible with mobile (iPhone/Android/Windows) and Web. With player-side support for segmented streaming, one format has become important amongst content providers to deliver videos across mobile and web.

bmcldrevHTTP Live Streaming, or HLS, is a video delivery protocol originally implemented by Apple as part of their iOS operating system. Zencoder’s API-driven video encoding solution makes preparing content for HLS easy, but there are still a few things worth knowing that will help content providers optimize content for their individual situation. This blog provides an overview of important information about HLS video playback in browsers and mobile devices.

If you not aware of Zencoder and its encoding API, please go through our last post on Zencoder Cloud Encoding.

HTTP Live Streaming

HTTP Live Streaming (also known as HLS) is an HTTP-based media streaming communications protocol implemented by Apple Inc. as part of their QuickTime and iOS software. It works by breaking the overall stream into a sequence of small HTTP-based file downloads, each download loading one short chunk of an overall potentially unbounded transport stream. As the stream is played, the client may select from a number of different alternate streams containing the same material encoded at a variety of data rates, allowing the streaming session to adapt to the available data rate. At the start of the streaming session, it downloads an extended M3U (m3u8) playlist containing the metadata for the various sub-streams which are available.

Since its requests use only standard HTTP transactions, HTTP Live Streaming is capable of traversing any firewall or proxy server that lets through standard HTTP traffic, unlike UDP-based protocols such as RTP. This also allows content to be delivered over widely available CDNs.

HLS also specifies a standard encryption mechanism using AES and a method of secure key distribution using HTTPS with either a device specific realm login or HTTP cookie which together provide a simple DRM system. Later versions of the protocol also provide for trick mode fast-forward and rewind and integration of subtitles. upLynk has also added the AES scrambling and base-64 encoding of the DRM content key with a 128-bit device specific key for registered commercial devices together with a sequential initialization Vector for each chunk to their implementation of the standard.

As the name implies, HLS transmits data over HTTP, which allows for various improvements over traditional streaming protocols like RTP or RTMP, including:

  • Reduced infrastructure costs
  • “Cacheability” at CDNs and other HTTP caching infrastructure
  • Less threat from proxy and firewall restrictions
  • Real-time optimization through client heuristics (adaptive bitrate)
  • Built-in redundancy
  • Simple HTML5 player implementation

 HTTP Live (Segmented) Streaming and Playlists

HTTP Live Streaming make it easy to adapt your video or audio stream to a user based on their available bandwidth, switching between streams in mid-play as the transfer speed changes. Pulling all of the pieces can be confusing, so we’ll start with how to create a few simple segmented files for different bandwidths and a playlist to provide to users on iOS devices so that they can play them.

The Segmented Files

We’ll start off by making several segmented files, each targeted at a given bandwidth. With these outputs Zencoder creates a manifest file and segmented video files, which allow the player to download each file individually as it plays the video.

{
    "audio_bitrate": 128,
    "audio_sample_rate": 44100,
    "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
    "decoder_bitrate_cap": 1500,
    "decoder_buffer_size": 4000,
    "filename": "sample-640k.m3u8",
    "max_frame_rate": 30,
    "public": false,
    "type": "segmented",
    "video_bitrate": 600,
    "width": 400,
    "format": "ts"
}

The m3u8 file created by the above output is a segmented file. Requesting this output, plus the others described below, will create multiple segmented files targeted at different bandwidths.

This output has a type of “segmented“, a format of “ts“, and defaults to a video codec of “h264“. Each segmented output will have a manifest file in M3U8 format created as its main output, and the segments will be named like the filename of the output. The segmented files will be uploaded along-side the manifest, and named with “-00001” suffixes before the extension. These outputs can be used as-is for standard streaming, simply by referencing the .m3u8 file from the client, and ensuring the segment files are available from the same location as the manifest. Multiple .m3u8 file can also be referenced by a playlist to support adaptive streaming.

The Playlist

You can create adaptive-bitrate playlists yourself. To create the playlist, just add another output to your job with a type of “playlist” and specify one or more streams with a path (that is relative to the playlist file’s location) and the bitrate of the stream in kbps (resolution and codecs can also be specified). The stream information will be formatted into a playlist and uploaded just like any other output.

This playlist file references the segmented outputs created above, specifying the bandwidth at which each should be played. Loading this file with a device that supports HLS will allow the stream to be adjusted as available bandwidth changes.

The first entry in the streams will be used when a user opens the file and is used as part of a test to determine which stream is most appropriate. The order of the other entries is irrelevant.

{
    "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
    "filename": "playlist.m3u8",
    "public": 1,
    "streams": [
        {
            "bandwidth": 440,
            "path": "sample-440k.m3u8"
        },
        {
            "bandwidth": 640,
            "path": "sample-640k.m3u8"
        },
        {
            "bandwidth": 240,
            "path": "sample-240k.m3u8"
        },
        {
            "bandwidth": 150,
            "path": "sample-150k.m3u8"
        },
        {
            "bandwidth": 64,
            "path": "sample-64k.m3u8"
        }
    ],
    "type": "playlist"
}

The Complete Picture

Multiple outputs of different bitrates may be specified and referenced by the playlist.

{
    "input": "s3://bluepi-original.s3.amazonaws.com/sample.mov",
    "output": [
        {
            "audio_bitrate": 56,
            "audio_sample_rate": 22050,
            "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
            "filename": "sample-64k.m3u8",
            "format": "aac",
            "public": 1,
            "type": "segmented"
        },
        {
            "audio_bitrate": 56,
            "audio_sample_rate": 22050,
            "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
            "decoder_bitrate_cap": 300,
            "decoder_buffer_size": 800,
            "filename": "sample-240k.m3u8",
            "max_frame_rate": 15,
            "public": 1,
            "type": "segmented",
            "video_bitrate": 200,
            "width": 400,
            "format": "ts"
        },
        {
            "audio_bitrate": 56,
            "audio_sample_rate": 22050,
            "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
            "decoder_bitrate_cap": 600,
            "decoder_buffer_size": 1600,
            "filename": "sample-440k.m3u8",
            "public": 1,
            "type": "segmented",
            "video_bitrate": 400,
            "width": 400,
            "format": "ts"
        },
        {
            "audio_bitrate": 56,
            "audio_sample_rate": 22050,
            "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
            "decoder_bitrate_cap": 900,
            "decoder_buffer_size": 2400,
            "filename": "sample-640k.m3u8",
            "public": 1,
            "type": "segmented",
            "video_bitrate": 600,
            "width": 480,
            "format": "ts"
        },
        {
            "audio_bitrate": 56,
            "audio_sample_rate": 22050,
            "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
            "decoder_bitrate_cap": 1500,
            "decoder_buffer_size": 4000,
            "filename": "sample-1040k.m3u8",
            "public": 1,
            "type": "segmented",
            "video_bitrate": 1000,
            "width": 640,
            "format": "ts"
        },
        {
            "audio_bitrate": 56,
            "audio_sample_rate": 22050,
            "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
            "decoder_bitrate_cap": 2250,
            "decoder_buffer_size": 6000,
            "filename": "sample-1540k.m3u8",
            "public": 1,
            "type": "segmented",
            "video_bitrate": 1500,
            "width": 960,
            "format": "ts"
        },
        {
            "audio_bitrate": 56,
            "audio_sample_rate": 22050,
            "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
            "decoder_bitrate_cap": 3000,
            "decoder_buffer_size": 8000,
            "filename": "sample-2040k.m3u8",
            "public": 1,
            "type": "segmented",
            "video_bitrate": 2000,
            "width": 1024,
            "format": "ts"
        },
        {
            "base_url": "s3://bluepi-encoded.s3.amazonaws.com/",
            "filename": "playlist.m3u8",
            "public": 1,
            "streams": [
                {
                    "bandwidth": 2040,
                    "path": "sample-2040k.m3u8"
                },
                {
                    "bandwidth": 1540,
                    "path": "sample-1540k.m3u8"
                },
                {
                    "bandwidth": 1040,
                    "path": "sample-1040k.m3u8"
                },
                {
                    "bandwidth": 640,
                    "path": "sample-640k.m3u8"
                },
                {
                    "bandwidth": 440,
                    "path": "sample-440k.m3u8"
                },
                {
                    "bandwidth": 240,
                    "path": "sample-240k.m3u8"
                },
                {
                    "bandwidth": 64,
                    "path": "sample-64k.m3u8"
                }
            ],
            "type": "playlist"
        }
    ]
}

Creating MP4 and HLS Outputs Together

Generally you don’t need only HLS encoding, because some flash players support RTMP protocol which is secured and with the help of CDN it transfer data in segments so you need MP4 files too. For that purpose you have to create/encode your original videos in MP4 as well as HLS outputs, so in this section we will discuss how you can generate both types of outputs together in one request.

With Zencoder you can encode your videos to MP4 and then also repackage those as HLS outputs, through a process we call “transmuxing” (rather than transcoding). Transmuxing will repackage existing MP4 videos into the MPEG TS segments necessary for HTTP Live Streaming (HLS), without having to re-encode the video files. Using dependent outputs you can create your H.264 files plus the segmented files all in a single job, producing faster turnaround at lower cost; transmuxed outputs are charged at 1/4 of the cost of encoding. The resulting job creates 6 outputs – 2 charged at your account’s regular rate, 3 transmuxed files at 1/4 of the encoding cost, and 1 playlist, which is free.

For this, you’ll create a single job with two main sets of outputs (plus the playlists). The first set of outputs will be normal H.264/AAC MP4 files with a few special settings to allow them to work as source files for HLS outputs. The second set of outputs will use those MP4 files as “source” inputs, transmuxing their content to HLS outputs. Since these are dependent outputs, they will wait until their corresponding source finishes before being scheduled to process.

The key part of the process is using source, a new option in V2 of the Zencoder API. Source tells an output to use the file created by another output on the job for processing, instead of the input file. In this situation, Zencoder will create H.264 files based on the input file at requested bitrates. As each of those outputs finishes, a segmented version will then be created from the output.

We have already discussed normal encoding to MP4 output in  last blog and HLS encoding as explained above, so the merged request will look something like as follows:

{
    "input": "s3://bluepi-original.s3.amazonaws.com/sample.mov",
    "outputs": [
        {
            "label": "high",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/high.mp4",
            "audio_bitrate": 160,
            "audio_sample_rate": 48000,
            "height": 720,
            "width": 1280,
            "max_frame_rate": 30,
            "video_bitrate": 5000,
            "h264_level": "3.1",
            "format": "mp4",
            "decoder_bitrate_cap": 3000,
            "decoder_buffer_size": 8000,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "95%",
                    "y": "-92%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark-desktop.png"
                }
            ],
            "public": false
        },
        {
            "source": "high",
            "label": "hls-high",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-high/hls-high.m3u8",
            "public": false
        },
        {
            "label": "main",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/main.mp4",
            "audio_bitrate": 128,
            "audio_sample_rate": 44100,
            "height": 480,
            "width": 640,
            "max_frame_rate": 30,
            "video_bitrate": 2000,
            "h264_level": "3",
            "format": "mp4",
            "decoder_bitrate_cap": 2250,
            "decoder_buffer_size": 6000,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "95%",
                    "y": "-92%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark-desktop.png"
                }
            ],
            "public": false
        },
        {
            "source": "main",
            "label": "hls-main",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-main/hls-main.m3u8",
            "public": false
        },
        {
            "label": "baseline",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/baseline.mp4",
            "audio_bitrate": 128,
            "audio_sample_rate": 44100,
            "height": 320,
            "width": 480,
            "max_frame_rate": 30,
            "video_bitrate": 1500,
            "h264_level": "3.1",
            "format": "mp4",
            "decoder_bitrate_cap": 1500,
            "decoder_buffer_size": 4000,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "92%",
                    "y": "-88%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark-desktop.png"
                }
            ],
            "public": false
        },
        {
            "source": "baseline",
            "label": "hls-baseline",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-baseline/hls-baseline.m3u8",
            "public": false
        },
        {
            "label": "baselinelow",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/baselinelow.mp4",
            "audio_bitrate": 128,
            "audio_sample_rate": 44100,
            "height": 240,
            "width": 320,
            "max_frame_rate": 30,
            "video_bitrate": 768,
            "h264_level": "1.3",
            "format": "mp4",
            "decoder_bitrate_cap": 600,
            "decoder_buffer_size": 1600,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "92%",
                    "y": "-88%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark.png"
                }
            ],
            "public": false
        },
        {
            "source": "baselinelow",
            "label": "hls-baselinelow",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-baselinelow/hls-baselinelow.m3u8",
            "public": false
        },
        {
            "label": "3gp",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/3gp.mp4",
            "audio_bitrate": 24,
            "audio_sample_rate": 16000,
            "height": 240,
            "width": 320,
            "max_frame_rate": 15,
            "video_bitrate": 192,
            "h264_level": null,
            "format": "mp4",
            "decoder_bitrate_cap": 300,
            "decoder_buffer_size": 1200,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "93%",
                    "y": "-90%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark-mobile.png"
                }
            ],
            "public": false
        },
        {
            "source": "3gp",
            "label": "hls-3gp",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-3gp/hls-3gp.m3u8",
            "public": false
        },
        {
            "source": "baselinelow",
            "label": "hls-audio-only",
            "type": "segmented",
            "format": "aac",
            "segment_video_snapshots": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-audio-only/hls-audio-only.m3u8",
            "copy_audio": "true",
            "skip_video": "true",
            "public": false
        },
        {
            "type": "playlist",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/playlist.m3u8",
            "streams": [
                {
                    "path": "hls-high/hls-high.m3u8",
                    "bandwidth": 2040
                },
                {
                    "path": "hls-main/hls-main.m3u8",
                    "bandwidth": 1540
                },
                {
                    "path": "hls-baseline/hls-baseline.m3u8",
                    "bandwidth": 640
                },
                {
                    "path": "hls-baselinelow/hls-baselinelow.m3u8",
                    "bandwidth": 240
                },
                {
                    "path": "hls-3gp/hls-3gp.m3u8",
                    "bandwidth": 64
                },
                {
                    "path": "hls-audio-only/hls-audio-only.m3u8",
                    "bandwidth": 56
                }
            ],
            "public": false
        }
    ]
}

Protecting Videos on Devices with HLS Encryption

Zencoder can perform AES encryption of video files. Our implementation is designed to be compatible with HTTP Live Streaming (HLS) outputs, but can be used to encrypt any video file similarly. When used for segmented outputs, only the segment files will be encrypted (leaving the manifest/playlist files readable, as expected). For HLS outputs, the necessary key/IV data will be added to the M3U8 manifest file, and encryption key files will be included with the other deliverable files if applicable.

Encryption is turned on when encryption_key, encryption_key_url, encryption_method, or encryption_key_rotation_period are specified.

Setting Default Description
encryption_method none Set the encryption method to use for encrypting.
encryption_key none Set a single encryption key to use rather than having Zencoder generate one
encryption_key_url none Set a URL to a single encryption key to use rather than having Zencoder generate one
encryption_key_rotation_period none Rotate to a new encryption key after a number of segments
encryption_key_url_prefix none Prepend key URLs with the passed string
encryption_iv none Set an initialization vector to use when encrypting
encryption_password none Sets a password to use for generating an initialization vector

The Whole picture

Now let’s see whole picture with Mp4 and HLS encoding and with Encryption.

{
    "input": "s3://bluepi-original.s3.amazonaws.com/sample.mov",
    "outputs": [
        {
            "label": "high",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/high.mp4",
            "audio_bitrate": 160,
            "audio_sample_rate": 48000,
            "height": 720,
            "width": 1280,
            "max_frame_rate": 30,
            "video_bitrate": 5000,
            "h264_level": "3.1",
            "format": "mp4",
            "decoder_bitrate_cap": 3000,
            "decoder_buffer_size": 8000,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "95%",
                    "y": "-92%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark-desktop.png"
                }
            ],
            "public": false
        },
        {
            "source": "high",
            "label": "hls-high",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-high/hls-high.m3u8",
            "public": false,
            "encryption_method": "aes-128",
            "encryption_key": "a5xxxxxxxxxx63"
        },
        {
            "label": "main",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/main.mp4",
            "audio_bitrate": 128,
            "audio_sample_rate": 44100,
            "height": 480,
            "width": 640,
            "max_frame_rate": 30,
            "video_bitrate": 2000,
            "h264_level": "3",
            "format": "mp4",
            "decoder_bitrate_cap": 2250,
            "decoder_buffer_size": 6000,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "95%",
                    "y": "-92%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark-desktop.png"
                }
            ],
            "public": false
        },
        {
            "source": "main",
            "label": "hls-main",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-main/hls-main.m3u8",
            "public": false,
            "encryption_method": "aes-128",
            "encryption_key": "a5xxxxxxxxxx63"
        },
        {
            "label": "baseline",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/baseline.mp4",
            "audio_bitrate": 128,
            "audio_sample_rate": 44100,
            "height": 320,
            "width": 480,
            "max_frame_rate": 30,
            "video_bitrate": 1500,
            "h264_level": "3.1",
            "format": "mp4",
            "decoder_bitrate_cap": 1500,
            "decoder_buffer_size": 4000,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "92%",
                    "y": "-88%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark-desktop.png"
                }
            ],
            "public": false
        },
        {
            "source": "baseline",
            "label": "hls-baseline",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-baseline/hls-baseline.m3u8",
            "public": false,
            "encryption_method": "aes-128",
            "encryption_key": "a5xxxxxxxxxx63"
        },
        {
            "label": "baselinelow",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/baselinelow.mp4",
            "audio_bitrate": 128,
            "audio_sample_rate": 44100,
            "height": 240,
            "width": 320,
            "max_frame_rate": 30,
            "video_bitrate": 768,
            "h264_level": "1.3",
            "format": "mp4",
            "decoder_bitrate_cap": 600,
            "decoder_buffer_size": 1600,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "92%",
                    "y": "-88%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark.png"
                }
            ],
            "public": false
        },
        {
            "source": "baselinelow",
            "label": "hls-baselinelow",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-baselinelow/hls-baselinelow.m3u8",
            "public": false,
            "encryption_method": "aes-128",
            "encryption_key": "a5xxxxxxxxxx63"
        },
        {
            "label": "3gp",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/3gp.mp4",
            "audio_bitrate": 24,
            "audio_sample_rate": 16000,
            "height": 240,
            "width": 320,
            "max_frame_rate": 15,
            "video_bitrate": 192,
            "h264_level": null,
            "format": "mp4",
            "decoder_bitrate_cap": 300,
            "decoder_buffer_size": 1200,
            "h264_reference_frames": 1,
            "h264_profile": "main",
            "forced_keyframe_rate": "0.1",
            "decimate": 2,
            "watermarks": [
                {
                    "x": "93%",
                    "y": "-90%",
                    "url": "s3://bluepi-images.s3.amazonaws.com/videos/logo-watermark-mobile.png"
                }
            ],
            "public": false
        },
        {
            "source": "3gp",
            "label": "hls-3gp",
            "type": "segmented",
            "format": "ts",
            "copy_audio": "true",
            "copy_video": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-3gp/hls-3gp.m3u8",
            "public": false,
            "encryption_method": "aes-128",
            "encryption_key": "a5xxxxxxxxxx63"
        },
        {
            "source": "baselinelow",
            "label": "hls-audio-only",
            "type": "segmented",
            "format": "aac",
            "segment_video_snapshots": "true",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/hls-audio-only/hls-audio-only.m3u8",
            "copy_audio": "true",
            "skip_video": "true",
            "public": false,
            "encryption_method": "aes-128",
            "encryption_key": "a5xxxxxxxxxx63"
        },
        {
            "type": "playlist",
            "url": "s3://bluepi-encoded.s3.amazonaws.com/playlist.m3u8",
            "streams": [
                {
                    "path": "hls-high/hls-high.m3u8",
                    "bandwidth": 2040
                },
                {
                    "path": "hls-main/hls-main.m3u8",
                    "bandwidth": 1540
                },
                {
                    "path": "hls-baseline/hls-baseline.m3u8",
                    "bandwidth": 640
                },
                {
                    "path": "hls-baselinelow/hls-baselinelow.m3u8",
                    "bandwidth": 240
                },
                {
                    "path": "hls-3gp/hls-3gp.m3u8",
                    "bandwidth": 64
                },
                {
                    "path": "hls-audio-only/hls-audio-only.m3u8",
                    "bandwidth": 56
                }
            ],
            "public": false
        }
    ]
}

 

References

By Barkat Dhillon

Introduction to MetoerJS

In case you happened to miss our webinar on #MeteorJS you can now watch it at leisure.

Meteor is an awesome real-time JS framework that is about to hit 1.0 soon. It makes application development a breeze. JS is used in both client and Server thereby providing a seamless developer experience. Built on top of NodeJS and MongoDB, Meteor uses a unique mechanism to keep data on the Client and the Server synchronized.

Watch to learn more.

MerteorJS Fundamentals from Pronam on Vimeo.

Introduction to AWS Landscape

We did a webinar on AWS Landscape and the different services provided by Amazon. If you could not attend please find the video. Our webinar’s schedule is available at Events section of our website. Feel free to register.

New Approach In Software Development Outsourcing

Change is the only constant in offshore product development. What was the prevailing attitude a few years ago has given way to new approach and preferences. While early adopters of the model had cost cutting at the top of their mind, the organizations now demonstrate a new long term strategic approach.

agile

Four key trends that make up the approach are:

  1. improving the quality of software
  2. obtaining higher business flexibility
  3. persuading expertise that is missing in-house or is too costly to acquire;
  4. seeking superior scalability

Software development companies today need to excel in several facets of business such as operations, product development, sales, marketing, support, etc. They must have core competency in product development though their being well-versed with other processes helps as well. If the firm has partnered with a partner that can develop world class products, they can be real asset.

When outsourcing product development, the objective of several organizations is to boost productivity and software quality by tapping into the skills, expertise and technology that they may lack in-house.

Internet driven environment today requires constant vigilance and responsiveness on the part of business organizations. Many find choosing an outsourcing partner a better option to flex up and down in sync with market conditions and business requirements.

Experience has taught organizations that there are times when obtaining certain expertise from external sources is more efficient and more affordable rather than hiring people. The perception of outsourcing service providers is also changing. Many enterprises view their outsourcing partners as vital, acting as extensions of their IT and business operations.

Outsourcing facilitates quick accomplishment of the scale needed for tackling opportunities and challenges presented by changing environment. 

Quality of relationship between client and vendor is key to the success of outsourcing. It does create a difference in service, quality, cost and other performance indicators. If you are looking for offshore software development in India (http://www.bluepi.in/), zero in on a company that can act for you as a trusted partner. It will help you get optimum value off the engagement.

Learn how to implement Infinite Scroll Pagination in AngularJS

I bet you have seen and pretty used to the old fashioned pagination that looks something like below:
old-pagination

The pagination style above suffers from a significant UX issues. The user has to guess on what page is the result he is looking for. He clicks and clicks in the hope of finding the right page and finally gets frustrated and leaves your site.

Also, you are asking too many inputs from user to traverse between the pages. Previous, Next, Page Numbers .. turns out though this is something that users have really gotten used to.

As a developer too,  such a pagination scheme is very easy to implement as it loads limited data .

How will you react if I improve user’s  experience by changing multiple pages to a single page, user does not need to click between pages because all the records are being shown in single page now & I guess that makes him happy!

At the same time you don’t need to return entire data set at once which means you can still send same chunk of data records as you were sending earlier, but bonus point here is that you need to send data again which you sent earlier, i.e. if you have already sent records 11-20 and user wants to see those again, request will not come to server, client will handle it internally.

Let’s do the magic

Let’s first discuss our design approach to achieve such functionality where we can show paginated records but in same page. Please refer following diagram:

Design Approach

In above diagram you can refer first column where browser displays only few records i.e. first page as in older pagination approach. We will observe scroll down event by the user and when we find out that user has reached bottom of record list then we will initiate an AJAX request to get next page record, do not replace current page’s record but append to current record list. Now user will have a experience that he is scrolling down same page, but you are sending data in chunks with the help of AJAX request.

I am going to implement this design in AngularJS, but you can implement this as per your framework as well. I will take an example of Courses where application is showing list of courses.

Back-end Service Contract

You need to define service method which can provide you data accordingly. We need 2 method as follows:

  1. getCourseCount() and it will get me total no. of courses exists in system so that application can stop loading more once it reaches count limit.
    url: /v1/courses?count
  2. getCourses(offset:Number, limit:Number) and it return list of records start from offset and upto limit records.
    Url: /v1/courses/offset/:offset/limit/:limit

Directive to Load Records on Scroll Down

Now application needs a directive which you can reuse throughout the application for all the pagination. This directive should be generic enough and should take a callback method as parameter so that each page can call its specific logic to load next chunk of records.


angular.module('app.directives.pvScroll', [])
.directive('pvScrolled', function() {
return function(scope, elm, attr) {
var raw = elm[0];

elm.bind('scroll', function() {
if (raw.scrollTop + raw.offsetHeight >= raw.scrollHeight) {
scope.$apply(attr.pvScrolled);
}
});
};
});

Above directive will bind an event ‘scroll’ to current element and as soon it reaches bottom of current element, it will invoke function passed in as parameter while applying this directive on an element.

Template to Render List

Use above coded directive in your view template so that it bind ‘scroll’ event with your container. Container could be anything either your full browser window or just a fraction of page e.g. a div showing list of records, in this case you need to define a height of container so that whenever scroll reaches that height end it should fire an event and load more records. Apply following CSS to your container.


.courses-list{
position : relative;
width    : 100%;
height   : 500px;
overflow : hidden;
}

Assign above defined CSS class to container and bind pvScrolled directive to it as well as follows:


<div>
<div>
<span>
Showing {{resultList.length}} out of {{total}} records
</span>
</div>

<div>
<div class="courses-list"  pv-scrolled="loadMoreCourses()">
<div ng-class-odd="'course-box'" ng-class-even="'course-box-bgcolor'" ng-repeat="item in resultList">
<div>
<!-- render each course row here -->
</div>
</div>
<!-- This div will be showed whenever ajax request is active to bring more records -->
<div ng-show="loadingResult" class="span9 loading-inline alert alert-info">
</div>
<!-- if server does not has more records to load then following div will inform the user that it is end of records -->
<div ng-show="!loadingResult && (resultList.length == total)" class="span9 pv-message">
<legend>
No more results to display.
</legend>
</div>
<!-- if no records found then following div will be showed -->
<div ng-show="!loadingResult && resultList.length == 0" class="span9 pv-message">
<legend>
No record found!!!
</legend>
</div>

</div>
</div>
</div>

Controller to Load Records on scroll down

In Controller we need to


angular.module('app.search.courses', [
'ui.state',
'app.services.course.courseService'
])
.config(['$stateProvider', function (stateProvider) {
stateProvider
.state('courses', {
url: '/courses',
views: {
'headerView@': {
templateUrl: 'templates/header.tpl.html'
},
'': {
templateUrl: 'courses/templates/courses.tpl.html',
controller: 'CourseCtrl'

},
'footerView@': {
templateUrl: 'templates/footer.tpl.html'
}
}
});
}])
.controller('CourseCtrl', ['$scope', 'CourseService',
function (scope, CourseService) {
scope.pagination = {
noOfPages: 1,
currentPage: 0,
pageSize: 10
};
scope.resultList = [];

scope.loadMoreCourses = function () {
if (scope.loadingResult) {
return;
}
if (scope.pagination.currentPage >= scope.pagination.noOfPages) {
return;
}
scope.pagination.currentPage = scope.pagination.currentPage + 1;
scope.offset = (scope.pagination.currentPage - 1) * scope.pagination.pageSize;
scope.limit = scope.pagination.pageSize;
scope.loadingResult = true;
scope.resultList = CourseService.courses.list({offset:scope.offset, limit:scope.limit});
scope.loadingResult = false;
};

scope.initializeResultList = function () {
CourseService.courses.count({}, function (count) {
scope.total = count;
scope.pagination.noOfPages = Math.ceil(count / scope.pagination.pageSize);
scope.loadMoreCourses();
});
}

scope.initializeResultList();
}]);

In the controller, we have declared a scope.pagination which captures pageSize i.e. limit or chuck size we are loading for every request, noOfPages i.e. total no. of requests required to traverse whole list and last is currentPage i.e. last accessed page so that if new request comes in to load more it will calculate which next chunk required to be loaded.

There are 2 methods defined too, loadMoreCourses() which will be called every time scroll reached bottom of container or when page loaded first time.This method has check that if method has already initiated load more AJAXrequest, it will not initiate next one.

Other is initializeResultList(), which will called one time at the time loading of controller and it will set total no of records and loads first chunk of records.

In html code, we have declared 3 different divs other than result list:

  1. Loading div: it will be shown whenever a AJAX request to load more records is in-progress.
    loadingMore
  2. No More Records to load: it will be shown whenever user has reached end of all the records available in the system. It means application will not load more records.
    noMoreRecords
  3. No Record found: If there is no record to show at all, then it will be show.
    noRecord

Please refer the similar implementation like we discussed above in screen shot of www.proversity.com.

Proversity - All Courses

Conclusion

By now you should be able to use this function in any of your AngularJS application or you can implement this design in your specific framework as well, if you have any additional suggestions or questions regarding this blog, feel free to leave a comment. Would love to share any ideas with you guys.

by Barkat Dhillon

Tapping Big Data for Increased Agility and Competitiveness in Business Operations

Stupendous growth of data volumes in recent years has been behind the rise of the phenomenon of big data, which has presented new challenges as well as new opportunities for business enterprises. Organizations can take advantage of the big data environment with analytic solutions to derive information about key business entities such as customers, products, suppliers, manufacturing processes and more.

Enterprise executives see exciting opportunities in big data. However, it is tempered by the continuing challenge to find solution which meets critical and frequently changing business requirements. This results in increased agility and competitiveness in business operations.

Data warehouse technology facilitates creation of database to capture, integrate, aggregate and store data from a range of sources. The data can be later queried, reported and analyzed for decision-making purposes.

A significant upfront development effort is required to set up the infrastructure for repeatable use of the data. The steps include creating –

  •  a business model of the data
  • logical data definition, called the schema
  • physical database design
  • extract/transform/load (ETL) process
  • business views of data for data delivery

SQL queries can be used on the data warehouse to retrieve information. As SQL is a declarative language, the user need not specify where to find the data, how to integrate it or how to manipulate it. This dramatically reduces the cost of creating an analytic application.

Data visualization turns complex data sets simple diagrams that can make unseen patterns and connections easy to understand. It helps navigate information glut and enables executives to see what they really want.

Enterprises have now recognized the value of representing research data in an innovative way. Data experts use visualization techniques to create charts that can be immensely helpful to marketing managers. Users are more likely to engage with data that has been turned into visual content.

Working with a company that is a part of Amazon AWS partner network , you can be sure of the quality of their data services. They can also be trusted for developing cloud applications. Zero in on a company that has ample technical resources to provide better services to clients.

Free Webinar: Introduction to Cloud Computing

Free Webinar for Cloud Computing Beginners

We are going to do a free webinar on the 15th of April.

In this hour long webinar we walk through the whys, whats and hows of the cloud computing. . We also look at the business drivers behind cloud computing. We discuss the benefits, common use cases, and financial & business benefits. We also do an  elementary demonstration of how to get started Amazon Web services.

Interested ? Click here to Register.