Exploiting Machine Generated Data - Chapter 1

Exploiting Machine Generated Data 

During the past few decades, the amount of machine data generated per millisecond has increased rapidly throughout the world. Machine-generated data is defined as, “information automatically generated by a computer process, application, or other mechanism without the active intervention of humans.” [- Wikipedia]. Web server logs, call detail records, financial instrument trades and network event logs are some examples for machine-generated data.

These data consist of a wealth of information that can be utilized for myriads of purposes. Therefore, computer scientists have been continuously working on developing highly scalable tools to extract this valuable insight from the data. But most of the time, the information extracted from these tools and algorithms are not presented in a way that can be used for decision making purposes of operational managers at an organization level in real time. However, if a real-time analytics system with an effective presentation level were to be built, it would certainly help operational managers of organizations to gain a deeper understanding of the uncovered information from machine data, reduce time to recognize vital events and take advantage of the live feeds and historical data to identify anomalies and make more effective decisions.

Web logs  as machine generated data 

In this article, web log records are taken as the machine generated data. These log files contain millions of records. A web log file records activity information when a web user submits a request to a web server. A log file can be located in three places; web servers, web proxy servers and client browsers . Each of these records contains an IP address, a HTTP status code, date, request address, number of bytes transmitted, user agent etc . These low-level data contain extremely valuable information such as security attacks, access information, bandwidth information, etc.

Moreover, by exploiting malicious user behavior patterns from these web logs, we can identify the possible security breaches that could occur and presenting them in real-time as “warnings”. This would help the system administrators to take measured actions at that moment to prevent these security breaches and enable them to drill down and analyze the historical security breach information. This, in turn, would assist them to distinguish system back-doors and loop holes. 

To be continued ..............

Setting up the development Environment for Moodle Mobile v 2.0

In this blog post I will mainly discuss on how I set up the development environment for Moodle Mobile v 2.0 in my computer. Actually this took approximately week and half . I am a windows user and when I tried setting up it in my windows 7 , it did not work. I tried , tried thinking that I would fly someday but could not actually take off until I shifted my track to Ubuntu. So I have made up computer to dual boot both the Ubuntu and Windows. In this post I will be explaining how to set up your development environment in Ubuntu 14.04 .

When setting up the environment, first we need to set up the Moodle web site instance in our machine. The newest version, 2.9 of moodle should be installed in your machine.

By going through this you will able to install the latest version (2.9) of Moodle in to your Ubuntu environment.

After installing and making Moodle Up and running, then you have to install  "the Moodle Mobile additional features plugin for Moodle 2.9 ". By going through this link , you will be able to successfully install it.

Then fork the repository from https://github.com/moodlehq/moodlemobile2 and clone the forked repository to the computer.

So now we are left with setting up the Moodle Mobile. By following the below described steps, you will be able to successfully set up Moodle Mobile in your Ubuntu environment.

Steps 

• Installing nodejs to the computer

Open the terminal and type sudo apt-get install nodejs command and then type sudo apt-get install npm to install npm. Then type sudo apt-get install nodejs-legacy as well. After the installation type node --version and it will out put the installed nodejs version as follows.

• Install ionic and cordova to the computer

Go to the terminal and use this command sudo npm install -g ionic to install ionic and sudo npm install -g cordova to install cordova. This will install ionic and cordova in your computer.

• Adding platforms

CD in to the moodlemobile2 directory and type ionic platform add ios android. This will add the platforms to the moodlemobile and will create two folders named android and IOS and a json file named platforms.json in the /moodlemobile2/platforms.

• Adding all the plugins listed in the package.json file

CD in to the moodlemobile2 directory and type ionic plugin add org.apache.cordova.globalization org.apache.cordova.file org.apache.cordova.inappbrowser org.apache.cordova.file-transfer org.apache.cordova.network-information org.apache.cordova.camera org.apache.cordova.media-capture

• Installing the dependencies

CD in to the moodlemobile2 directory and type npm install. This will install the dependencies which are listed in the package.json

• Installing 3rd party libraries .

CD in to the moodlemobile2 directory and type sudo npm install -g bower to install bower and then type bower install to add all the 3rd party libraries which are listed in the bower.json

• Creating JS file bundles

CD in to the moodlemobile2 directory and type sudo npm install -g gulp to install gulp and then type gulp to create JS file bundles.

• Running moodle mobile v 2.0

In order to work with moodle mobile v 2.0 we need to install google chromium browser. We can install it through terminal by typing sudo apt-get install chromium-browser . This will install chromium browser. We need to run this chromium browser with the flags , in order to access remote sites. This can be done by typing the following command. chromium-browser --allow-file-access-from-files --disable-web-security

Then type ionic serve to start the ionic server in localhost:8100 . Moodle mobile will be run on this.

So now we have finished setting up the development Environment for Moodle Mobile v 2.0. Now the coding begins. In the next post I will be discussing about the first component I started developing. Till then see you guys and Happy Coding. :D

Finalized Spec for the project - Javascript based SCORM 1.2 Player for the new moodle mobile app

In this blog post I will be mainly discussing about the specifications of the project that I am working on for GSOC 2015. When we look at the project in an overall view, this project ideally has two main components that need to be developed. First one is the additional web services for the Moodle site and the second one is the SCORM player. The web services will be developed by the developers in the moodle HQ . (By the time I publish this bog post they have already developed most of the web services).

Moodle Requirements 

SCORM Module Changes

 In order to make possible to play SCORM packages in online and offline modes, the Mobile app needs to download the complete zip SCORM package and extract to the local sdcard/storage system.

The Mobile app will notify the user if the SCORM package is supported, checking if the package was uploaded or downloaded via URL, if the “Allow to download complete SCORM packages” is enabled, and if there is enough free space in the device.

The Moodle guys have also created some new external web services for the moodle as well which are supposed to use in the SCORM mobile application. Following are the new web services.

mod_scorm_get_scorms_by_courses

Parameters: list of course ids

Returns: A list of SCORM packages module settings in the specified courses, including only the settings the user may see

mod_scorm_get_scorm_attempts

Parameters: scormid and userid

Returns: A list containing information of all the attempts done by the specified user id

This will be used to know the current attempt status for the user (last attempt, etc..)

mod_scorm_get_scorm_scoes

Parameters: scormid

Returns a list of scoes inside the specified scorm

mod_scorm_get_scorm_user_data

Parameters: userid, attempt

Returns all the user tracks for all the scorm scoes (including objectives and interactions)

This will be used for preparing the API runtime data

mod_scorm_insert_scorm_track

Parameters: scoid, userid, attempt, element, value

Inserts track information about a SCO

Returns: true or false (success or not)

Moodle Mobile Requirements

The SCORM player will be an AngularJS module integrated in a Mobile app addon. It will be responsible for:

1. Parsing the imsmanifest.xml file in order to display the available Organizations and Structures inside the package.
2. Displaying a basic SCORM player with a basic menu tree.
3. Implementing a basic runtime support in order to retrieve the SCOs tracking information (including objectives and interactions) and sending tracking information too.

The SCORM player will be an AngularJS player that will communicate with the Mobile app SCORM add-on via the SCORM runtime.

Moodle Mobile SCORM addon

A new addon for the SCORM module that will contain the AngularJS SCORM player and will perform the following tasks:

• Downloading and synchronizing the SCORM zip package
• Unzipping the package into the device persistent storage
• Populating the SCORM SCO’s default tracking information
• Storing and synchronizing tracking information with Moodle via Web Service

In order to do that, the SCORM player (AngularJS module) will have to communicate with the addon via the API runtime (functions LMSInitialize, LMSGetValue, LMSSetValue)

So the above features are finalized specification of this project. If you like to see some more features in the SCORM mobile player please feel free to leave them as comments.

Till next post -------------- Happy Coding ----------------------------------------

GSOC 2015 here I come .......

It has been nearly one year since I have posted a blog post but the exciting times ahead stimulated me to write this blog post. GSOC 2015 , in the beginning was a dream which was out of reach for me but with commitment, passion and guidance from a PIGGY (AKA Ura AKA Viduranga), I was able to finally make it a reality. I think I really do not have to explain much about GSOC, because it is the most awaited and the most competitive IT event of the year in any undergraduate who is majoring IT/CS. If you have not heard of GSOC before, go through this and continue the read. 

GSOC 2015, Moodle and Me

This the first time I have applied for GSOC and I am glad that I got selected because after this year I will not be eligible to participate as a student. So in the beginning I made sure that I give my 200 % percent to it. When GSOC 2015 projects are released, the first thing I did was to go through the organization lists and their projects to find a suitable project to apply. The organization which I chose was Moodle, the most famous word among any undergraduate throughout the world. Moodle is a learning platform which is designed to provide teachers, administrators and students an integrated learning environment. And the project was about developing a SCORM player for the new Moodle mobile app. This year three projects from moodle have been accepted for the GSOC 2015. 

Why Moodle ?

I have used Moodle from the very first day I entered university and still using it for academic purposes. I think moodle is a good platform to practice cohesive learning. It is being used by a lot of university students all around the world. Now the whole world has shifted to mobile and moodle has already moved on to mobile with their mobile app. Joining the developing process of the newest version of moodle mobile app which has the latest technologies and developing an important plugin for it will be an immense opportunity for me. That was the main reason behind selecting moodle and this project. 

What the hell is SCORM ????

When I first read the project description I actually did not get a clear idea because that was the first time I have ever heard of the word SCORM. But since the project was based on Javascript and because I am fan of Javascript and mobile applications, I thought of giving a shot for this. The first thing I did was starting research on SCORM. After some days I was able to find some several articles and PDFs about SCORM and got a basic understanding about that technology.

SCORM stands for Sharable Content Object Reference Model. It is a common standard which was introduced by Advanced Distributed Learning. These standards help that the SCORM based content can be used in any learning management system without any hassle. 

Road map

Having that basic understanding I followed the guidelines which were mentioned in the moodle website for GSOC participants. I followed each of these guidelines very clearly and had continuous communication with the mentor Juan Levya. It helped me to clarify a lot of doubts and to prepare my proposal as clear as possible as well. 

So the aim of this Project is to develop a Javascript based SCORM 1.2 Player for the new moodle mobile app which is based on ionic framework. This SCORM Player should be able to connect Moodle via Web Services for retrieving a SCORM package, unzips the file in the local storage. Retrieve additional information from the Moodle site and then launch the contents inside an iframe.

Now the community bonding period has started and the coding will start on the 25th of May. Hoping to get bonded with the Moodle Community in the coming weeks and wish me good luck for a summer full of codes. ;) :p 

From the next post onward I am hoping to give you all a brief understanding about the my GSOC proposal, the technical background of the project and the specifications.