Understanding and Applying Artificial Neural Networks : Engineering Perspective

Lasts week (15th November to 19th November), NITTTR (National Institute of Technical Teachers Training and Research) had organized a course on Artificial Neural Networks at Government College Of Engineering – Farmagudi, Goa. I was appointed as course faculty for this event.

Personally, I do not believe in concept of teaching. Humans are smart and intelligent animals and as such, they are capable of learning new things themselves based on their experience and observations. Thus, rather than providing a training, I preferred sharing my experience and knowledge on ANN with the members attending the course.

Considering the fact that this event was organized for staff of engineering colleges, the “Joint – Discussion” was titled, “Engineering Solutions With Artificial Intelligence Using A System Popularized As Artificial Neural Networks“.

Generally, we discuss Artificial Neural Networks either in theoretical perspective or in terms of purely scientific applications. However, the world of practical engineering is always different from theoretical or purely scientific perspective. The same applies to applications of ANN in perspective of modern digital engineering.

In this joint discussion, we spent first two days trying to understand the concept of artificial neural networks. To understand it practically, demonstrations were provided using the examples developed in JRuby. The code written for this purpose is now available on GitHub (The link is provided at the end of this post).

For next two days, all the participants were allowed to experiment with the JRuby code. Participants could play around with different training sets while modifying variables like learning rate, momentum, activation functions, number of hidden layers, number of neurons in hidden layers etc. They could also experiment with performing the same training using backpropagation algorithm and genetic algorithm. This helped them in understanding the challenges faced by engineers in practically implementing the solutions using ANN.

On the last day (Friday 19th, November 2010), we concluded the joint discussion by determining the capabilities and limitations of ANN based on the experience and knowledge obtained by participants in four days. At the end, a demo of OCR written in Javascript was given by Tejas Vernekar.

During the joint discussion, some of the alternative non-neural solutions for few problems were discussed as well. A ruby equivalent of spell checker (which uses statistical language processing) originally developed in python was also demonstrated. The ruby equivalent code is available in the code in GitHub Repository.

In case you are interested in JRuby code used during the joint discussion, you can download it from this GitHub repository.

Just Released: Maya Programming Language

Around one year ago, I had started experimenting with concepts of creating programming languages. I first started with building few DSLs (Domain Specific Languages) using Ruby and Boo. It was quite interesting to work on development of DSLs. While doing this, I learnt a lot about how the programming languages are developed.

Eventually, I thought of writing my own programming language. While I was in engineering college as a student, we had a subject on language translators. This subject taught us lot of theory on how languages are compiled or interpreted. However, we never got a chance to write even a simple programming language during our practicals. According to few, it was not possible to write one in one semester’s time.

That turned out to be my main motivation. I decided to write this programming language without using any compiler tools like LEX and YACC. I ended up writing my own lexical analyzer, the grammar parser and the language using the concepts taught to us in engineering college only. I worked on it daily for around an hour and within two weeks, I had the basic language ready.

Today, the code is released under GPL license. It’s available on GitHub at http://github.com/kamatanay/MayaLanguage

I hope it would be useful for students learning language translators.

Why did I name it Maya? Well, that’s the name of my sweetheart wife. Isn’t that a sweet name?

Error in ruby on rails documentation for ActionController::UrlWriter

In Ruby on rails, methods that generate urls from named routes are not globally accessible. For example, you can’t access them from console (script/console). If you want to use these methods from such places, then the rails documentation for ActionController::UrlWriter suggests two ways of doing it. According to this documentation, you can:

  1. Include ActionController::UrlWriter in your class
  2. Call the method directly on ActionController::UrlWriter

When I tried it out, only the first method worked. I was able to use methods generated from named routes in console after including ActionController::UrlWriter. However, it was not possible to call those methods on ActionController::UrlWriter. This looks like an issue with documentation to me.

It might be the case that the second method used to work in earlier version of rails. As rails is constantly being developed, some refactoring might have made the second method obsolete. I hope rails community will fix such issues in documentation soon.

RSpec Matchers: Be careful while testing boolean values

While testing methods that return boolean values in ruby (the ones that end in ‘?’), try to avoid following matchers:

method_returning_true?.should be_true

or

method_returning_false?should be_false

This is because, ‘be_true’ and ‘be_false’ matchers considers ‘nil’ to be false and anything other than ‘nil’ to be true. When we write methods in ruby which end with question mark (‘?’), intent is that the method will return boolean value. To ensure that our tests will always reflect the intent of code, use following to assert boolean values instead of using ‘be_true’ or ‘be_false’ matchers:

method_returning_true?.should == true

or

method_returning_false?.should == false

Simple equivalent of “With” statement in C#

Consider the following class in C#

	public class Person
	{
		private string name;
		private int age;
		
		public string Name{
			get {return name;}
			set { name = value; }
		}
		
		public int Age{
			get {return age;}
			set { age = value; }
		}
	}

If I want to set the value of Name and Age property on the instance of Person class, I’ll need to refer to that instance for every property I need to set in the code. For example:

var person = new Person();
person.Name = "Super Man";
person.Age = 30;

It would have been great if C# had an equivalent of VB’s “With..End” statement, where we could refer to the instance of Person class only once and then refer to properties only.

Today, I came across this post “mucking about with hashes…“, which shows how C# lambdas could be used as hashes. Using this concept, I implemented a simple extension method that simulates the behavior of VB’s “With..End” statement to some extent.

Here is the code for extension method:

	public static class MetaExtensions
	{	
		public static void Set(this object obj,params Func<string,object>[] hash){
				foreach(Func<string,object> member in hash){
					var propertyName = member.Method.GetParameters()[0].Name;
					var propertyValue = member(string.Empty);
					obj.GetType()
						.GetProperty(propertyName)
							.SetValue(obj,propertyValue,null);
				};
		}
	}

Using this extension method, we can set the value of properties on instance of Person class as follows:

var person = new Person();
person.Set(
	Name => "Super Man",
	Age => 30
);

Isn’t that cool?

Y-Combinator in C#

For last few days, I was trying to use lambda feature of C# to implement Y-Combinator. After few trial and errors, I was able to implement it in C# 3.5. I’m currently posting the code here and in my next blog, I’ll explain how I derived it.

In this code, Y-Combinator function, is used to implement anonymous recursive-factorial function called ‘factorial’.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace YCombinator
{
    class Program
    {
        delegate Func RecursiveFunction(RecursiveFunction f);

        static void Main(string[] args)
        {

            Func, Func>, Func> Y = (f) =>
            {
                RecursiveFunction function = (h) =>
                {
                    return (x) =>
                    {
                        return f(h(h))(x);
                    };
                };
                return function(function);
            };

            Func factorial = Y(function=>
            {
                return x =>
                {
                    return x == 0 ? 1 : x * function(x - 1);
                };
            });
            Console.WriteLine(factorial(5));
            Console.ReadLine();
        }
    }
}

Be careful while using ‘replaceAll’ and ‘replaceFirst’ methods of String class in Java

Most of the time, while trying to replace a substring in a given string, either all its occurrences or just the first one, java programmers tend to use ‘replaceAll’ and ‘replaceFirst’ methods provided by String class in Java. Java programmers like to use these methods to replace substrings as compared to ‘replace’ method of String class because of different reasons like:

  1. ‘replaceAll’ and ‘replaceFirst’ methods use regular expressions instead of character sequence. It is pre-assumed most of the time that using these methods will allow us to replace a given pattern in future, instead of just replacing a substring.
  2. ‘replaceAll’ and ‘replaceFirst’ methods are named such that they clarify the intent of their execution.

However, while using these methods, we need to be very careful while providing the string value which would replace a given substring or a pattern. Let’s understand why with the following example:

Run the following java code:

public class StringReplace {

public static void main(String[] args){
String stringValue = "test1 test2 test1 test3";
String replaceValue = "test";
stringValue = stringValue.replaceAll("test1", replaceValue);
System.out.print(stringValue);
}

}

This code works properly. But now try running the same code by changing the value of variable ‘replaceValue’ as:

String replaceValue = "$100";

This would result in an exception of type ‘IndexOutOfBoundException’. This is because the ‘$’ symbol is used to identify a group from the regular expression which is the first parameter of ‘replaceAll’ or ‘replaceFirst’ method. We can solve this problem by:

  1. Using ‘replace’ method: This would be the good choice if you want to replace a string literal and not a pattern.
  2. Escaping ‘$’ symbol: If you need to a use regular expression, and your pattern has no groups identified, then you can escape any group identification symbols from your replace string as shown below:

String replaceValue = java.util.regex.Matcher.quoteReplacement("$100");

Taking a close look at inheritance and closures in Javascript

Let’s play with inheritance and closures in Javascript. Take a look at the following javascript code:

function BaseClass(){
	var id=200;
	this.Id=function(){
		return id;
	}
}

function ChildClass(){
	var id=500;
}

ChildClass.prototype = new BaseClass();
var childObject=new ChildClass();

In Javascript, inheritance is achieved using prototype. When we try to access any member on Javascript object, it first tries to search for the member within that object. If the member is found, then its accessed otherwise it tries to search for that member in its prototype object.

We make BaseClass the parent class of ChildClass with the following instruction:

ChildClass.prototype = new BaseClass();

Notice that ChildClass does not have a method called ‘Id’. When we make a call to method ‘Id’ on childObject, it will see that ChildClass does not have ‘Id’ method and so will take it from its prototype object which is nothing but the object of BaseClass.

Based on this knowledge, what do you think will be the output of the following instruction?

childObject.Id()

If you think it should be 500, then you are wrong. The correct answer is 200. This happens due to what is known as Closures. According to a tutorial on javascript closures by Morris Johns:

“a closure is the local variables for a function – kept alive after the function has returned”

To understand how this works in our case, let’s try to understand how javascript actually executes a call to the method ‘Id’ on childObject.

As childObject does not have a method called ‘Id’, it takes that method from its prototype and then executes it. You can imagine that Javascript performs the following steps to execute a call to the method ‘Id’ on childObject.

childObject.Id = ChildClass.prototype.Id;
childObject.Id();

But does its mean that in Javascript, it’s not possible to override the values from base class/object at all? Well, It is possible by making a small change to our code. Javascript has a keyword called ‘this’. The ‘this’ keyword points to the object in which the method being called is present. For more information on ‘this’ keyword, refer to the article ‘The this keyword’.

In our current code, we are defining a variable using keyword ‘var’ which makes it local to the function in which it is defined. Due to this, closure comes into picture and call to the method ‘Id’ always prints the value from the BaseClass. To solve this problem, we need to refer to the variable ‘id’ using ‘this’ keyword. This will tell our method to use the value from the object in which the method has been placed. In this way, when the method is placed in ChildClass object, it will print the value of ‘id’ from ChildClass and not from BaseClass. Thus, the modified code which allows overriding is given below:

function BaseClass(){
	this.id=200;
	this.Id=function(){
		return this.id;
	}
}

function ChildClass(){
	this.id=500;
}

ChildClass.prototype = new BaseClass();
var childObject=new ChildClass();

Binding HTML Form Fields To Javascript Objects

Consider that I have a object called ‘person’ with a property called ‘Name’ which returns me the name of a particular person. I want to assign this object property to a form element such that:

  1. The form element will display the value of object property (In our case, the value of person.Name)
  2. If I change the value in form, then it should automatically get assigned to the object property

In C#, it could be done by using the following piece of code:

nameTextBox.DataBindings.Add("Text", person, "Name");

I really like the way DataBinding works and would really like have something like this when I’m working on web applications to bind a object property to input elements in HTML forms. So I decided to write a small javascript that would allow me to bind object properties to form elements. For example, I wanted to write something like this:

<input type="text" object="person" property="firstName" />

Thanks to the power of Javascript and prototype library, I was able to use data binding in html forms using code:

function initializeFormBinding()
{
	var formElements = document.getElementsByTagName('input');
	$A(formElements).each(function(formElement)
				{
					Element.extend(formElement);
					initializeFormElement(formElement);
				});
}
 
function initializeFormElement(formElement)
{
	if (!(formElementHasObjectAttribute(formElement) &amp;&amp; formElementHasPropertyAttribute(formElement)))
		return;
	var objectName = getAttributeValue(formElement,'object');
	var propertyName = getAttributeValue(formElement,'property');
	window.eval('formElement.value = '+objectName+'.'+propertyName);
	window.eval('formElement.onchange=function(){ '+objectName+'.'+propertyName+' = formElement.value; }');
}
 
function formElementHasObjectAttribute(formElement)
{
	return getAttributeValue(formElement,'object')!=null;
}
 
function formElementHasPropertyAttribute(formElement)
{
	return getAttributeValue(formElement,'property')!=null;
}
 
function getAttributeValue(element, attributeName)
{
	return element.readAttribute(attributeName);
}

Lets see how to use this: First, create a javascript object. For example:

var person = {};
person.firstName = '';
person.lastName = '';
person.age = '';
person.country = '';

After that call initializeFormBinding() function using onLoad attribute of body tag. For example:

<body onLoad="javascript:initializeFormBinding();">

Now, in each of your input elements, put the name of the object in ‘object’ attribute and name of property in ‘property’ attribute as follows:

<form>
First Name:
<input type="text" object="person" property="firstName" /><br>
Last Name:
<input type="text" object="person" property="lastName" /><br>
Age:
<input type="text" object="person" property="age" /><br>
Country:
<input type="text" object="person" property="country" /><br>
</form>

Thats it…. If you initialize properties of person object, you would see those values in respective form elements. On the other hand, if you modify the value in a particular input element then it would get reflected in the corresponding object property. Anybody is free to use this code in their applications.

Download a small example application..

Using JSON in Alfresco WebScripts

In my current project, we are using Alfresco for the content repository and all the extra functionalities required are developed using Webscripts. While working on one functionality, I had to serialize javascript objects to JSON and also deserialize it. For doing this, there is a nice javascript code on Alfresco Wiki for converting the object to JSON and vice versa. But to use it, you need to make some slight modifications to the code. The original code doesn’t generate proper JSON code and also has a syntax error in method to obtain the javascript object back from the JSON string. So here is the proper code for handling JSON:

/*
   json.js
(modified 2006-09-07): added support for RHINO
(modified 2006-05-02): json.parse, json,stringify added
  USAGE:
  var jsObj = JSON.parse(jsonStr);
  var jsonStr = JSON.stringify(jsObj);
*/
if (!this.json) {
  var json = (function () {
      var m = {
              'b': 'b',
              't': 't',
              'n': 'n',
              'f': 'f',
              'r': 'r',
              '"' : '\"',
              '': ''
          },
          s = {
              array: function (x) {
                  var a = [], b, f, i, l = x.length, v;
                  for (i = 0; i < l; i += 1) {
                      v = x[i];
                      f = s[typeof v];
                      if (f) {
                          v = f(v);
                          if (typeof v == 'string') {
                              a[a.length] = v;
                              b = true;
                          }
                      }
                  }
                  return '['+a.join()+']';
              },
              'boolean': function (x) {
                  return String(x);
              },
              'null': function (x) {
                  return "null";
              },
              number: function (x) {
                  return isFinite(x) ? String(x) : 'null';
              },
              object: function (x) {
                  if (x) {

                      if (x instanceof Array) {
                          return s.array(x);
                      }
                      if (x.hashCode) return s.string(+x.toString());

                      var a = [], b, f, i, v;
                      for (i in x) {
                          v = x[i];
                          f = s[typeof v];
                          if (f) {

                              v = f(v);

                              if (typeof v == 'string') {
                                  a.push(""+s.string(i)+':'+ v+"");
                                  b = true;
                              }
                          }
                      }
                      return '{'+a.join()+'}';
                  }
                  return 'null';
              },
              string: function (x) {
                  if (/["x00-x1f]/.test(x)) {
                      x = x.replace(/([x00-x1f\"])/g, function(a, b) {
                          var c = m[b];
                          if (c) {
                              return c;
                          }
                          c = b.charCodeAt();
                          return 'u00' +
                              Math.floor(c / 16).toString(16) +
                              (c % 16).toString(16);
                      });
                  }
                  return '"' + x + '"';
              }
          };

     return {
        parse: function(s) {
           try {
              return !(/[^,:{}[]0-9.-+Eaeflnr-u nrt]/.test(s.replace(/"(.|[^"])*"/g,""))) && eval('(' + s + ')');
           } catch (e) {
              return false;
           }
        },
        stringify: s.object
     };
  })();
}

Here is how to use this code to serialize a javascript object:

	o={};
	o.name = "Name";
	o.id = 1;
	j = json.stringify(o);

Also, to obtain the object back from JSON string, you can use following method:

	x = json.parse(j);