tag:blogger.com,1999:blog-25083401561468342342024-03-14T10:06:00.398+01:00Software Craftsman's Blog by Marcin PieciukiewiczPractical software developmentMarcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.comBlogger17125tag:blogger.com,1999:blog-2508340156146834234.post-15043495127519175462016-09-05T21:03:00.000+02:002016-09-06T09:41:42.888+02:00Generating Java JKS keystore file from PEM filesLately I had to configure SSL/https certificate for our Java based application. To do so I needed Java Key Store file (*.jks), that was not delivered by our certificate provider. Instead I had available only binary or text based certificate files. This means I had to generate JKS from those files. Below is the instruction how to do this:<br />
<br />
You will need:<br />
<ol>
<li>Certificate file for your domain</li>
<li>Private key for this certificate</li>
<li>Intermediate certificates</li>
<li>Root CA certificate</li>
</ol>
<div>
For certificate files I've used *.pem format as it is common and easy to handle (those are Base64 encoded text files).</div>
<div>
<br /></div>
<div>
So to generate JKS file from PEM files do this:</div>
<div>
<br /></div>
<div>
1. Bundle pem files into single file, The order of those files is very important, and should be like this: first main certificate, then intermediate certificates in the same order as they were on the page, and rootCA at the end)</div>
<div>
<br /></div>
<pre class="brush:bash">$ cat certificate.pem inter1.pem inter2.pem inter3.pem inter4.pem rootCA.pem > bundle.pem
</pre>
<br />
<div>
2. Generate pkcs12 (from bundle.pem and private.key)</div>
<div>
<br /></div>
<pre class="brush:bash">$ openssl pkcs12 -export -inkey private.key -in bundle.pem -out yourdomain.com.p12 -name yourdomain.com
</pre>
<br />
<div>
3. Generate *.jsk file</div>
<div>
<br /></div>
<pre class="brush:bash">$keytool -importkeystore -destkeystore yourdomain.com.jks -srckeystore yourdomain.com.p12 -srcstoretype PKCS12 -alias yourdomain.com
</pre>
<br />
<div>
In the end you will have yourdomain.com.jks file that can be imported to you application.</div>
Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com3tag:blogger.com,1999:blog-2508340156146834234.post-59996828133444901902013-09-15T12:09:00.000+02:002013-09-17T20:32:33.322+02:00Functional approach to Exception handling in ScalaDid you know that in functional programming world there is a criticism of throwing exceptions? It seems that it would break a functional programming principle that a function have to always return a value. I'm not sure that this approach is the best for all situations, but I can imagine some cases when it might be good. Especially because Scala dumped the idea of checked exceptions.<br />
<br />
So what we could do if we would like not to throw the exception when the exceptional situation occurs? In example, when we receive a zero as denominator in division operator. In standard approach we could throw IllegalArgumentException, but what to do in a purely functional approach? We could return a <span class="code">Not-A-Number</span> value or an <span class="code">Infinity</span> value. But that would not be the best approach if this exceptional situation should be handled somehow. Also there is a risk that this value would be passed further in an application and that might cause a difficult to detect problem in other part of the code.<br />
<br />
More reliable solution would be to return an info about the error. We could reuse the <span class="code">IllegalArgumentException</span>, as it is an object that nicely describes the problem and, also, it can hold a stack trace, which can be useful for debugging. So let's look at the example code:<br />
<pre class="brush:scala">def divide(a:Double, b:Double) = {
if (b == 0) {
new IllegalArgumentException("Cannot divide by zero!")
} else {
a / b
}
}
</pre>
That might work, but what type returns the divide function? In this case Scala compiler we'll choose <span class="code">Any</span> as closest common ancestor. That means, a user of our function we'll have to check returned type and manually cast a result to a <span class="code">Double</span>. You can imagine that this is inconvenient, and sometimes might cause <span class="code">ClassCastException</span> when user won't predict correctly all possible types that might be returned.<br />
<br />
Some literature suggests usage of <span class="code">Either[A,B]</span> class from standard Scala's library. Basically this is an extension of an Option idea. <span class="code">Either</span> object can hold a value in one of it's two fields: <span class="code">left</span> and <span class="code">right</span>. This can be leveraged to be useful in exception handling if we assume that we'll always put correct result in right field and error in left field. This way we could have more control over returned type, and we could handle an exception using pattern matching. I would recommend to get familiar with <span class="code">Either</span> class as it can be useful in some situations, but I also think that it is not a best solution to handle exceptional situations. First, for my taste, it is too general and it requires a convention that must be known by every user. I would like to propose a different approach, inspired by <span class="code">Either</span> class, but with more specific purpose.<br />
<br />
My solution is based on a new <span class="code">Result</span> class which would be a holder of a function result value, or an exception, if required. Here is it's definition:<br />
<pre class="brush:scala">abstract class Result[A, E <: Throwable] {
def isSuccess: Boolean
def isFailure: Boolean
def get: A
def getOrDefault(default: => A): A
def error: E
}</pre>
As you can imagine this class can have two basic states implemented as two case classes, <span class="code">Success</span> and <span class="code">Failure</span>:
<br />
<pre class="brush:scala">final case class Success[A, E <: Throwable](result: A) extends Result[A, E] {
override def isSuccess = true
override def isFailure = false
override def get: A = result
override def error: E = throw new IllegalAccessException("It's a success, not a failure")
override def getOrDefault(default: => A): A = result
}
final case class Failure[A, E <: Throwable](err: E) extends Result[A, E] {
override def isSuccess = false
override def isFailure = true
override def get: A = throw err
override def error: E = err
override def getOrDefault(default: => A): A = default
}</pre>
Those two are very simple implementation of abstract <span class="code">Result[A, E]</span> class. Before we go into details of this implementation let's look how it can be used in our <span class="code">divide</span> function:<br />
<pre class="brush:scala">class DivideByZeroException extends IllegalArgumentException("Cannot divide by zero")
def divide(a: Double, b: Double): Result[Double, DivideByZeroException] = {
if (b == 0) {
Failure(new DivideByZeroException)
} else {
Success(a / b)
}
}
</pre>
I hope it doesn't require a lot of explanation. Our modified function is returning a <span class="code">Result[Double, DivideByZeroException]</span> type which will be a Failure containing an exception if <span class="code">b</span> is equal zero, or a Success containing the result of computation in other cases.
<br />
Let's look closer on a methods provided by <span class="code">Result</span> class:
<br />
<ul>
<li><span class="codeBold">isSuccess</span> and <span class="codeBold">isFailure</span> - those methods simply allow us to distinguish between <span class="code">Success</span> and <span class="code">Failure</span> objects</li>
<li><span class="codeBold">get</span> - this methods returns a result value in the case of a <span class="code">Success</span> object, or it <b>throws the inner exception</b> object in the case of an <span class="code">Failure</span> object</li>
<li><span class="codeBold">error</span> - this method <b>returns an exception</b> object that was contained inside <span class="code">Failure</span> object, or it throws an <span class="code">IllegalAccessException</span> in the case of a <span class="code">Success</span> instance</li>
<li><span class="codeBold">getOrDefault(default: => A)</span> - this method is similar to get method, so it returns a result value in the case of <span class="code">Success</span> object, but in the case of <span class="code">Failure</span> instead of throwing an exception it returns a value passed as default (if default is an expression it is evaluated only if needed)</li>
</ul>
<div>
This set of methods, and the fact that <span class="code">Success</span> and <span class="code">Failure</span> are case classes, provides us with a convenient way of handle results returned from functions. Let's look at some examples where function succeedes:</div>
<pre class="brush:scala">scala> divide(10, 2)
res = Success(5.0)
scala> divide(10, 2).get
res = 5.0
scala> divide(10, 2).isSuccess
res = true
scala> divide(10, 2).isFailure
res = false
scala> divide(10, 2).getOrDefault(27)
res = 5.0
scala> divide(10, 2).error
java.lang.IllegalAccessException: It's a success, not a failure // IllegalAccessException was thrown
</pre>
And now the case when function returns a <span class="code">Failure</span>:<br />
<pre class="brush:scala">scala> divide(10, 0)
res = Failure(java.lang.IllegalArgumentException: Cannot divide by zero)
scala> divide(10, 0).get
DivideByZeroException: Cannot divide by zero // DivideByZeroException was thrown
scala> divide(10, 0).isSuccess
res = false
scala> divide(10, 0).isFailure
res = true
scala> divide(10, 0).getOrDefault(27)
res = 27.0
scala> divide(10, 0).error
res = DivideByZeroException: Cannot divide by zero // DivideByZeroException was returned, not thrown
</pre>
Those were simple, method's use cases, here you can look how it is possible to handle exceptions now:<br />
Simple if checking for success:
<br />
<pre class="brush:scala">val divideResult = divide(10, 2)
if (divideResult.isSuccess) {
println("Result of division is " + divideResult.get)
} else {
println("You shouldn't divide by zero")
}
</pre>
Do you remember the C times when negative number have meant an error?
<br />
<pre class="brush:scala">val result = divide(10, 0).getOrDefault(-1)
</pre>
And this is probably most convenient way of doing this, with a usage of pattern matching:
<br />
<pre class="brush:scala">divide(20, 5) match {
case Success(x) => println("Result of division is " + x)
case Failure(err) => println("Something went wrong: " + err.getMessage)
}</pre>
<b><br /></b>
<b>Conclusion</b><br />
This approach to exception handling is only a proposition how it could be done if we would like to limit the throwing of the exceptions. There are people that believe in clean functional approach to programming and I think that they should be listened and understood. For me, dilemma between throwing an exception or returning it reminds me discussion about checked vs unchecked exceptions, that still doesn't have a winner.<br />
<br />
I think that presented approach could be used when an exceptional situation is part of a business, ie. when parsing user's input. In this case we have to provide a response for any data user provided, even this corrupted. On the other hand this approach shouldn't be used to handle exception caused by internal error in our software, ie. parsing a rest request from other part of our system. In this case there is probably nothing smart to do, other than to fail gracefully, so it is best to throw an exception that will be handled by our application server.<br />
<br />
I'm sure that with proper care both approaches have the justification for existence, even in one application.
<br />
<br />
<b>Edit</b><br />
As Konrad mentioned, Scala 2.10 is equipped with scala.util.Try class, which works almost the same as my Result class. That's great news, you can give it a try with no cost o creating custom class, and if you would like, you can use it in production code already.Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com4tag:blogger.com,1999:blog-2508340156146834234.post-50323088535393137182013-08-26T18:15:00.000+02:002013-08-28T21:51:36.237+02:00Writing a simplified JSON parser in Clojure<div>
<div>
Today I want to
write a step by step example of creating a parser for JSON. What is
extraordinary, I'll use a <a href="http://clojure.org/">Clojure</a> to
do that. This is because I have to learn Clojure for my current job
and I think parsing problem is very good for this, due it's
complexity and usefulness.</div>
<div>
<br /></div>
<div>
What is <a href="http://clojure.org/">Clojure</a>?
Basically it is a programming language that is a dialect of another
language called Lisp. And it is targeted for Java Virtual Machine.
Here is some minor description from <a href="http://clojure.org/">clojure.org</a>:</div>
<div>
<br /></div>
<div>
<i>Clojure is
predominantly a functional programming language, and features a rich
set of immutable, persistent data structures.</i></div>
<div>
<br /></div>
<div>
For those that aren't
familiar with Clojure this article might be too difficult, because
Clojure has a tremendously different syntax than ie. Java or Scala.
In that case it would be better to start with some other tutorial
that covers Clojure basics or with some book, ie. "Practical
Clojure". I found that book to be a very good, easy to read and
a comprehensive introduction to the language.</div>
<div>
<br /></div>
<div>
Enough introduction, let's
go to our parser problem.</div>
<div>
<br /></div>
<div>
<b>Define parser
requirements</b></div>
<div>
We want to write a JSON
parser. Why JSON? I've chosen it because of a couple of reasons.
First it is very popular in many commercial usages (ie. RESTful
services or NoSQL databases), on the second hand it has very simple
syntax so it will be good for training.</div>
<div>
<br /></div>
<div>
JSON syntax is
defined on a page <a href="http://www.json.org/">www.json.org</a>.
So basically in JSON we could have 6 different value types:</div>
<ol>
<li><div>
Object - wrapped in
curly brackets, properties separated by commas and values separated
from properties names separated with colon. ie. <span class="code">{name:"Marcin",
surname:"Pieciukiewicz"}</span></div>
</li>
<li><div>
Array - wrapped
in square brackets, values separated with commas, ie.
<span class="code">["dog","cat","cow"]</span></div>
</li>
<li><div>
String - sequence of
characters wrapped in double quotes, ie. <span class="code">"dog"</span></div>
</li>
<li><div>
Number</div>
</li>
<li><div>
Boolean - <span class="code">true</span> or
<span class="code">false</span></div>
</li>
<li><div>
Null - <span class="code">null</span></div>
</li>
</ol>
<div>
For our purpose I would
like to make some <b>simplifications to the syntax</b>, so we
could focus on general parsing problems:</div>
<ol>
<li><div>
No white characters
are allowed (outside a string)</div>
</li>
<li><div>
Number can be non
negative integer only</div>
</li>
<li><div>
String cannot contain
double quotes and escape character won't be supported, so "\"
will be treated as normal character</div>
</li>
<li><div>
Parsed JSON is always correct</div>
</li>
</ol>
<div>
All those simplifications
are introduced so our parser could be simple enough to well fit in
this article.</div>
<div>
<br /></div>
<div>
<b>Parsing based on
sequence traversing</b></div>
<div>
Clojure provides us with a
powerful sequence mechanism and we will use it to traverse through a
JSON string. You can imagine that we want to have a cursor over a
JSON string that will be used to read subsequent characters, and it
will only move forward (from the beginning of the json string, to the
end).</div>
<div>
To do so we'll have to
utilize two commands:</div>
<pre class="brush:text">(first some-string)</pre>
<div>
and</div>
<pre class="brush:text">(rest some-string)</pre>
<div>
<span class="code">first</span> command simply returns first element from the given sequence,
and <span class="code">rest</span> command returns a sequence of the elements after the first
element. We'll use this commands extensively mainly in recursive
operations. It's important to notice that <span class="code">first</span> and <span class="code">rest</span> functions
work on sequence passed as parameter, in our case this functions
implicitly converts passed string into a sequence.</div>
<div>
<br /></div>
<div>
To understand better how
we'll utilize those functions this is a simple example of printing
every character from stream in separate line:</div>
<pre class="brush:text">(defn print-in-lines [text]
(if (not (empty? text))
(do
(println (first text))
(recur (rest text)))))</pre>
This function first checks if given text is not empty and, if so,
it prints out the first character of text and then calls itself
recursively with text parameter stripped of the first character.<br />
<div>
<br /></div>
<div>
<b>Detect element type</b></div>
<div>
Because our parser will
traverse json string only once we need to detect what kind of element
is going to be next. To do that we'll define a set of functions that
will check the first character from json sequence and compare it with
a character from JSON syntax. Ie. we want to be able to detect
beginning of an object. To do so we'll define <span class="code">is-object-opening?</span>
function:</div>
<pre class="brush:text">(defn is-object-opening? [json] (= \{ (first json)))</pre>
<div>
As you can see (which might be hard in Clojure ;)) this function
takes a json sequence and checks if it's first character is an
opening curly bracket “{“. And in the end it returns boolean
value. Also we would like to know when this object ends, so let's
define function to detect closing curly bracket:</div>
<pre class="brush:text">(defn is-object-ending? [json] (= \} (first json)))</pre>
<div>
This is very simple, so
let's define this kind of functions for array and string detection:</div>
<pre class="brush:text">(defn is-array-opening? [json] (= \[ (first json)))
(defn is-array-ending? [json] (= \] (first json)))
(defn is-string-opening? [json] (= \" (first json)))
(defn is-string-ending? [json] (= \" (first json)))</pre>
<div>
Also we'll need to detect integers, boolean values and nulls. To do
that we'll assume that integer value starts with a digit from 0 to 9,
boolean value starts with lowercase 't' (for true) or lowercase 'f'
(for false), and null starts with lowercase 'n'. Now we can define
functions that will detect those values:</div>
<pre class="brush:text">(defn is-boolean-opening? [json] (or (= \t (first json)) (= \f (first json))))
(defn is-null-opening? [json] (= \n (first json)))
(defn is-digit? [json]
(and
(>= (int (first json)) (int \0))
(<= (int (first json)) (int \9))))
(defn non-digit? [json]
(not (is-digit? json)))</pre>
<div>
As you can see I haven't
defined functions that detects the end of boolean or null value. That
is because we exactly know what values can start with 't', 'f' or 'n'
and in particular how long they are. And we'll use this knowledge
later.</div>
<div>
For integers instead of
creating <span class="code">is-integer-opening?</span> and <span class="code">is-integer-ending?</span> I've created more
generally named functions that do exactly the same thing.</div>
<div>
<br /></div>
<div>
<span style="color: black;"><b>Parsing
primitive values - number, string, boolean and null</b></span></div>
<div>
<span style="color: black;">When
we know what is going to be next in the sequence we want to parse it.
We'll start with a basic types, object and array we'll leave for
later.</span></div>
<div>
<span style="color: black;">So we
need to create a function for each type that will return a parsed
value, and, evenly important, a new sequence that is pointing just
after parsed value. You can think about it as returning new cursor
position in json text.</span></div>
<div>
<br /></div>
<div>
<span style="color: black;"><u>Parsing
string.</u></span></div>
<div>
<span style="color: black;">String
parsing is about reading our json stream until we occur double quote
symbol. We'll do that recursively (tail-recursive). What is
important, we assume that json sequence passed to this function
doesn't start with opening double quote, because it was "consumed"
when we checked value type. This is parse-string function
implementation (in first call string is initialized to an empty
string ""):</span></div>
<pre class="brush:text">(defn parse-string [json string]
(if (is-string-ending? json)
[string (rest json)]
(recur (rest json) (str string (first json)))))</pre>
<div>
<span style="color: black;">As
you can see we check first character of json to see if we should
finish parsing. If so we return string (which is our accumulator of
read value) and the remaining json sequence. We've used rest function
to skip closing double quote character. Otherwise, if we haven't
found double quote, we remove first character of passed sequence and
concatenate current string value with first character in current
sequence and make a recursive call to <span class="code">parse-string</span> function.</span></div>
<div>
<br /></div>
<div>
<span style="color: black;"><u>Parsing
number.</u></span></div>
<div>
<span style="color: black;">Number
parsing is very similar to string parsing. We simply read subsequent
character until it's not a digit, and meanwhile we increase an
accumulated value (in first call number have to be initialized to
zero):</span></div>
<pre class="brush:text">(defn char-to-number [character]
(- (int character) (int \0)))
(defn parse-number [json number]
(if (non-digit? json)
[number json]
(recur (rest json) (+ (char-to-number (first json)) (* number 10)))))</pre>
<div>
<span style="color: black;">To
convert a digit character into corresponding numerical value I've
created <span class="code">char-to-number</span> function that simply casts character into
ASCII integer value and calculate difference between that value and
integer value of '0' character.</span></div>
<div>
<br /></div>
<div>
<span style="color: black;"><u>Parsing
boolean.</u></span></div>
<div>
<span style="color: black;">To
parse a boolean we'll just have to check the first letter. If it is
't' the value is 'true', if the first letter is 'f' the boolean value
is 'false'. Now implementation of parser function is very simple:</span></div>
<pre class="brush:text">(defn parse-boolean [json]
(if (= \t (first json))
[true (drop 4 json)]
[false (drop 5 json)]
)
)</pre>
<div>
<span style="color: black;">We
don't have to detect where the value string ends because we know that
<span class="code">true</span> is 4 characters long, and <span class="code">false</span> is 5
characters long, so we'll just drop corresponding number of
characters from json sequence.</span></div>
<div>
<span style="color: black;">To be
clear, we could do that because we assumed that json we are parsing
is correct.</span></div>
<div>
<br /></div>
<div>
<span style="color: black;"><u>Parsing
null.</u></span></div>
<div>
<span style="color: black;">Parsing
<span class="code">null</span> value is even simpler than parsing a boolean value. Because
we'll call our <span class="code">parse-null</span> function after we detected null value,
we'll only need to skip 4 characters from json sequence and return
nil:</span></div>
<pre class="brush:text">(defn parse-null [json]
[nil (drop 4 json)]
)</pre>
<div>
<span style="color: black;"><b>Call
a parse function for a detected type</b></span></div>
<div>
<span style="color: black;">Now
we have to combine our detection of element type with proper parse
function. To do that we'll create <span class="code">parse-value</span> function (because
everything is a value ;)):</span></div>
<pre class="brush:text">(declare parse-object parse-array)
(defn parse-value [json]
(cond
(is-digit? json) (parse-number json 0)
(is-boolean-opening? json) (parse-boolean json)
(is-null-opening? json) (parse-null json)
(is-string-opening? json) (parse-string (rest json) "")
(is-object-opening? json) (parse-object (rest json) {})
(is-array-opening? json) (parse-array (rest json) [])
:else (println "Incorrect character?")))</pre>
<div>
<span style="color: black;">Also
we've declared <span class="code">parse-object</span> and <span class="code">parse-array</span> functions, so <span class="code">parse-value</span>
function would compile. We'll implement those functions later.</span></div>
<div>
<br /></div>
<div>
<span style="color: black;"><b>Parse
array</b></span></div>
<div>
<span style="color: black;">Array
parsing will be a little more complex. Basically we need to read
elements separated by comma character, until we occur closing square
bracket “]” that is closing the array. We'll do that by
recursively calling our <span class="code">array-parse</span> function and in each run it will
read one element of the array. This is the implementation:</span></div>
<pre class="brush:text">(defn drop-comma-if-first [json]
(if (= \, (first json))
(rest json)
json))
(defn parse-array [json array]
(if (is-array-ending? json)
[array (rest json)]
(let
[value-result (parse-value (drop-comma-if-first json))
value (nth value-result 0)
rest-json (nth value-result 1)]
(recur rest-json (conj array value)))))
</pre>
<div>
As you can see we simply stop the recursion when the first element of
json stream is “]” character (<span class="code">is-array-ending?</span> function). If not, we have
to read element value and to do that we'll use parse-value function.
There is also <span class="code">drop-colon-if-first</span> function used - it simply skips the
first colon in json sequence if it is present, this is the case when
we want to read second or subsequent element from array and we need
to skip separator.</div>
<div>
<br /></div>
<div>
Also in <span class="code">parse-array</span> function we use let statement so
we can acquire result of <span class="code">parse-value</span> function that is a vector
containing parsed value, and also the new json stream that points
just after parsed value. After that we simply call <span class="code">parse-array</span>
function recursively with new json stream and array with added value
we've just parsed (we've used <span class="code">conj</span> function to do that).<br />
<br /></div>
<div>
<b>Parse object</b></div>
<div>
Object parsing is
very similar to array parsing, with the difference that we also need
to parse the key name of the value. To do that we have to introduce a
method that will be responsible for parsing of that key:</div>
<pre class="brush:text">(defn parse-property-name [json name]
(if (= \: (first json))
[name json]
(recur (rest json)
(str name (first json)))))</pre>
It works alike <span class="code">parse-string</span>
function, but instead of looking for double quote it is looking after
colon to check if the name has ended. Next, let's look at the
implementation of <span class="code">parse-object</span> function:<br />
<pre class="brush:text">(defn parse-object [json object]
(if (is-object-ending? json)
[object (rest json)]; rest to skip closing '}'
(let
[property-name-result (parse-property-name (drop-comma-if-first json) "")
property-name (nth property-name-result 0)
value-json (rest (nth property-name-result 1))
value-result (parse-value value-json)
value (nth value-result 0)
rest-json (nth value-result 1)]
(recur rest-json (assoc object property-name value)))))</pre>
As you can see it has very
similar construction to <span class="code">parse-array</span> function. In <span class="code">let</span> statement it
calls <span class="code">parse-property-name</span> function, remembers it's result, then
calls <span class="code">parse-value</span> function. Also it uses <span class="code">assoc</span> function to add new
property to constructed object.<br />
<br />
<b>Main parser function</b><br />
The last thing to do is to
create a main function that can be access point to our parser:<br />
<pre class="brush:text">(defn parse-json [json]
(nth (parse-value json) 0))</pre>
It simply calls <span class="code">parse-value</span>
function and from it's result extracts first element which is a
parsed element. That's all.<br />
<br />
<b>Summary</b><br />
I hope that this example showed
that Clojure is a good way to solve more complex problems, such as
string parsing. It might not be a best language to do so, but it is
not a great struggle to work with it either. For sure Clojure will
force you to think differently than Java and this might be it's
greatest advantage.<br />
<br />
Source code with full solution
is available on gist.github.com: <a href="https://gist.github.com/marpiec/6341987">json_parser.clj</a>.</div>
Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com3tag:blogger.com,1999:blog-2508340156146834234.post-31319604042796196042013-08-21T21:00:00.000+02:002013-08-21T22:29:24.871+02:00Scalatra - How to setup with Maven and JettyRecently I was inspired by a colleague to dig into AngularJS and Scalatra web application stack, so today I'll continue with first steps in digging into Scalatra.<br />
<br />
Description from <a href="http://www.scalatra.org/">www.scalatra.org</a>:<br />
<i>Scalatra is a simple, accessible and free web micro-framework.</i><br />
<i>It combines the power of the JVM with the beauty and brevity of Scala, helping you quickly build high-performance web sites and APIs.</i><br />
<i><br /></i>
And when you look at examples you can get feeling that this might be a great framework to help you develop your applications quickly:<br />
<pre class="brush:scala">package com.example.app
import org.scalatra._
class HelloWorldApp extends ScalatraFilter {
get("/") {
<h1>Hello, {params("name")}</h1>
}
}</pre>
<br />
So let's start playing with it by creating a sample project that will use Maven as the build tool and a Jetty server as our application's container.<br />
<br />
<b>1. Define build process and application dependencies</b><br />
To start we need to create main <span class="code">pom.xml</span> file required by maven. It will be placed in the main directory we've created for our project.<br />
First we need to configure it to properly compile Scala language sources. I have described this process in article <a href="http://mpieciukiewicz.blogspot.com/2013/03/maven-configuration-example-for-scala.html" target="_blank">Maven configuration example for Scala</a>. So you should follow instructions from that article.<br />
<br />
When this is done we'll add support for Scalatra. To do that we'll need to include <span class="code">org.scalatra:scalatra</span> dependency in <span class="code">pom.xml</span>. Also let's include library <span class="code">org.scalatra:scalatra-specs2</span> that will add support for writing tests in the future. So add this part of code inside <span class="code"><dependencies></span> tag (To be clear, I've used a Scalatra dependencies prepared for Scala 2.10, this is the reason for weird suffixes in <span class="code">artrifactId</span>):<br />
<pre class="brush:xml"><!-- Scalatra -->
<dependency>
<groupId>org.scalatra</groupId>
<artifactId>scalatra_2.10</artifactId>
<version>2.2.0</version>
</dependency>
<dependency>
<groupId>org.scalatra</groupId>
<artifactId>scalatra-specs2_2.10</artifactId>
<version>2.2.0</version>
<scope>test</scope>
</dependency></pre>
<br />
Also Scalatra requires that Java Servlet API 2.5 be available during application compilation, so we'll add it in provided scope:<br />
<br />
<pre class="brush:xml"><!-- Servlet API -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
</dependency></pre>
<br />
That will be all to support Scalatra in our application, now let's add Jetty so we'll be able to run our application without any external application server. This only requires to add a plugin inside a <span class="code"><build> -> <plugins></span> tag:<br />
<pre class="brush:xml"><plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>9.0.2.v20130417</version>
</plugin></pre>
Now the configuration of our build process is complete. To summarize this is my complete <span class="code">pom.xml</span> for this example:<br />
<div style="height: 157px; overflow: auto;">
<pre class="brush:xml"><?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>pl.marpiec</groupId>
<artifactId>scalatratest</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>war</packaging>
<properties>
<scala.lang.version>2.10.1</scala.lang.version>
<scala.maven.version>2.10.1</scala.maven.version>
<scalatra.version>2.2.0</scalatra.version>
<jetty.version>9.0.2.v20130417</jetty.version>
</properties>
<repositories>
<repository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>https://oss.sonatype.org/content/groups/scala-tools/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>https://oss.sonatype.org/content/groups/scala-tools/</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<!-- Scala -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.lang.version}</version>
</dependency>
<!-- Scalatra -->
<dependency>
<groupId>org.scalatra</groupId>
<artifactId>scalatra_2.10</artifactId>
<version>${scalatra.version}</version>
</dependency>
<dependency>
<groupId>org.scalatra</groupId>
<artifactId>scalatra-specs2_2.10</artifactId>
<version>${scalatra.version}</version>
<scope>test</scope>
</dependency>
<!-- Servlet -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>${scala.maven.version}</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<version>${jetty.version}</version>
</plugin>
</plugins>
</build>
</project></pre>
</div>
<br />
<b>2. Configure web application by web.xml file</b><br />
To be able to run our web application we need to create a standard Java web application description file, <span class="code">web.xml</span>. Let's create this file inside <span class="code">[Project Dir]\src\main\webapp\WEB-INF\</span> directory. It's content is very simple:<br />
<pre class="brush:xml"><?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee </pre>
<pre class="brush:xml"> http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
version="3.0">
<context-param>
<param-name>org.scalatra.LifeCycle</param-name>
<param-value>pl.marpiec.scalatratest.servlets.ScalatraBootstrap</param-value>
</context-param>
<listener>
<listener-class>org.scalatra.servlet.ScalatraListener</listener-class>
</listener>
</web-app></pre>
This will tell an application's server (Jetty in our case) that it have to call a <span class="code">ScalatraListener</span> when application starts. Also context parameter <span class="code">org.scalatra.LifeCycle</span> points to the <span class="code">ScalatraBootstrap</span> class that will be used by our application. This parameter is not required, but without it you will have to put <span class="code">ScalaBootstrap</span> class in root package, and that would be ugly. Also, you probably want to change the package name that is used to be align with your project ;).<br />
<br />
<b>3. Create your first Scalatra servlet</b><br />
So now let's create our first servlet that will send simple responses for <span class="code">GET</span> requests. This servlet might look like this:<br />
<pre class="brush:scala">package pl.marpiec.scalatratest.servlets
import org.scalatra.ScalatraServlet
/**
* @author Marcin Pieciukiewicz
*/
class MainServlet extends ScalatraServlet {
get("/") {
<html>
Hello to scalatra
</html>
}
get("/json-page") {
"<json-response>Hello to scalatra</json-resonse>"
}
}</pre>
As you can see this servlet will send an html response for the request of root application address, and it will send string containing json if <span class="code">/json-page</span> is requested.<br />
<br />
<b>4. Create ScalatraBootstrap class</b><br />
The last thing to do is to create a class mentioned earlier: <span class="code">ScalatraBootstrap</span>, and then bind our servlet with proper URL path:<br />
<br />
<pre class="brush:scala">package pl.marpiec.scalatratest.servlets
import org.scalatra._
import javax.servlet.ServletContext
class ScalatraBootstrap extends LifeCycle {
override def init(context: ServletContext) {
context.mount(new MainServlet, "/*")
}
}</pre>
<br />
This class is basicly starting point for scalatra configuration. In our case we've just added our server to be mapped for all url suffixes.<br />
<br />
And now our sample application is complete.<br />
<br />
<b>5. Test application</b><br />
To test this application we have to run it on servlet container, and as it was mentioned earlier we'll use Jetty runned as Maven plugin. So to start our server with our application just run:<br />
<pre class="brush:xml">mvn jetty:run</pre>
After a while a Jetty server will be running and listening on port <span class="code">8080</span> on <span class="code">localhost</span>. So just check the responses under <span class="code">http://localhost:8080</span>:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPPv1LRNoVWXjgB5mo4fTJRVc97EqfGAvu0iBDMyPtHFmNnGr9uFcoh6AMeW_5nc_0DF8F-PfqwnZ_5xHDpNfcNVcgcZYY8OkZ94ZeAbtbNlCZxOQP7Hk4lk9K6r0-2qbF9ekaQVgIIU3q/s1600/scalatra1.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiPPv1LRNoVWXjgB5mo4fTJRVc97EqfGAvu0iBDMyPtHFmNnGr9uFcoh6AMeW_5nc_0DF8F-PfqwnZ_5xHDpNfcNVcgcZYY8OkZ94ZeAbtbNlCZxOQP7Hk4lk9K6r0-2qbF9ekaQVgIIU3q/s1600/scalatra1.png" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
And <span class="code">http://localhost:8080/json-page</span>:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhM1OWzb1zJzSFO8oIXFDJVc1nFfA2AuAkkXC3NyDo4M_R6uuAK6s6v4psuijYQHOU6A3PleG7keq6oAF1h6KUjiNcWZwxBgNQExo0ur1_7Pixgx8sj-sMzWtQvx79yCpa8Gao9IIy6rp0g/s1600/scalatra2.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhM1OWzb1zJzSFO8oIXFDJVc1nFfA2AuAkkXC3NyDo4M_R6uuAK6s6v4psuijYQHOU6A3PleG7keq6oAF1h6KUjiNcWZwxBgNQExo0ur1_7Pixgx8sj-sMzWtQvx79yCpa8Gao9IIy6rp0g/s1600/scalatra2.png" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com0tag:blogger.com,1999:blog-2508340156146834234.post-67306557788110712982013-08-18T18:15:00.001+02:002013-08-18T18:15:47.871+02:00AngularJS example - simple calculatorRecently my attention was focused on a very interesting JavaScript framework named <a href="http://angularjs.org/">AngularJS</a>. It is a framework created by Google to augment creation of rich browser-based application. In this article I would like to present an example of application created with AngularJS that will use some of the features provided by this framework. Of course I've just began to learn AngularJS so this article contains examples of basic concepts used by AngularJS.<br />
<br />
<b>1. Application concept</b><br />
Our application will be a very simple calculator, that provides the four basic mathematical operations, and it will work on integer numbers. The expected layout should be like that:<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSNyr_37K0a4MhYtrdSC-DAdtHsIt93ZT4QDJRDUhHU2T_-5TG6u3O_XPds5XNvwJnsCqBJz4vKCoxIhSfhakgGlTM9uA9glO3xQUnhk2W3n_KjVkgC9fbdxDPn4_2f1pqSgb8esdSUq1x/s1600/calc.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSNyr_37K0a4MhYtrdSC-DAdtHsIt93ZT4QDJRDUhHU2T_-5TG6u3O_XPds5XNvwJnsCqBJz4vKCoxIhSfhakgGlTM9uA9glO3xQUnhk2W3n_KjVkgC9fbdxDPn4_2f1pqSgb8esdSUq1x/s1600/calc.png" /></a><a href="http://www.blogger.com/blogger.g?blogID=2508340156146834234" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a></div>
<br />
<b>2. Download AngularJS framework</b><br />
AngularJS framework is distributed as a single JavaScript file and can be acquired on the <a href="http://angularjs.org/">angularjs.org</a> page. Just choose <b>stable </b>and <b>mininified </b>version of the file. It will be named angular.min.js. When I was writing this article the stable version was 1.0.7.<br />
<br />
<b>3. Create an application's structure</b><br />
Our application will be organized in this directory structure:<br />
<pre class="brush:text">Calculator Main project directory
|--lib
| |--js JavaScript 3rd party libraries
|--app
|--js JavaScript application source files
|--style CSS files
|--view HTML files
</pre>
When the application's structure is created, then we copy downloaded <span class="code">angular.min.js</span> file into <span class="code">Calculator/lib/js</span> directory.<br />
Then let's create basic source files that will define our application. We'll need three of them:<br />
<ul>
<li><span class="code">Calculator/index.html</span> - root view layer file of our application</li>
<li><span class="code">Calculator/app/js/app.js</span> - main source file, will put logic in this file</li>
<li><span class="code">Calculator/app/style/default.css</span> - main cascading style file</li>
</ul>
<b><br /></b>
<b>4. Create AngularJS application</b><br />
In next step we'll create an html skeleton for our application that will define basic html properties and include our source files. So the content of <span class="code">index.html</span> should look like that:<br />
<pre class="brush:html"><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html ng-app="calculatorApplication">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<script src="lib/js/angular.min.js" type="text/javascript"></script>
<script src="app/js/app.js" type="text/javascript"></script>
<link href="app/style/default.css" rel="stylesheet" type="text/css">
</head>
<body>
<div ng-view></div>
</body>
</html>
</pre>
Beside the standard html header there are two elements that are specific to AngularJS application:<br />
<ul>
<li><span class="code"><html ng-app="calculatorApplication"> </span></li>
It tells AngularJS that everything inside html tag is an AngularJS controlled application. And it is a module called "calculatorApplication".
<li><span class="code"><div ng-view></div></span></li>
This <span class="code">ng-view</span> attribute points to the element that will be a container, where AngularJS will put created view.
</ul>
As you can see a framework extends a standard HTML with its' own attributes that can be interpreted by AngularJS internals.<br />
<br />
After creating HTML structure of our application we have to add some behavior. We'll do that by editing <span class="code">app.js</span> file and adding there this part of code:<br />
<pre class="brush:javascript">var app = angular.module("calculatorApplication", []);
</pre>
This construct will create AngularJS module called <span class="code">calculatorApplication</span>. Empty square brackets stands for empty list of dependencies that our application will require.<br />
In the next step we would like to define some basic behavior for our application. Basically we want our application to have a home page that will have a welcome message and a link to our calculator.<br />
That page should look like that:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFxotpPShzfjKL_dSchnbXHLjO3FteZ6WiiR1cFGRGcghGsZowTejubhUqzU2PyNKvQx_TzbWOm9zaaZS_FKVbneiHH7fuoiSRyTmeRYIgLhn0OZ8CShwu94gyj0dX6Pf-aKKFYdbMKa8E/s1600/home.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgFxotpPShzfjKL_dSchnbXHLjO3FteZ6WiiR1cFGRGcghGsZowTejubhUqzU2PyNKvQx_TzbWOm9zaaZS_FKVbneiHH7fuoiSRyTmeRYIgLhn0OZ8CShwu94gyj0dX6Pf-aKKFYdbMKa8E/s1600/home.png" /></a></div>
As you can see the url for this page have an interesting <span class="code">#/home</span> suffix. Basically it is an address of our application subpage. This is a standard way to define subpages in AngularJS. Now we would like to bind this <span class="code">/home</span> suffix with some content. So let's add a controller definition of this page by changing <span class="code">app.js</span> content into this:<br />
<pre class="brush:javascript">var app = angular.module("calculatorApplication", []).config(function ($routeProvider) {
$routeProvider.when("/home", {
templateUrl: "app/view/home.html",
controller: "HomeController"
});
$routeProvider.otherwise({
redirectTo: "/home"
});
});
app.controller("HomeController", function () {
});
</pre>
A few things are happening in this part of code. First we've called config method and passed a function that will configure our application. This function takes a parameter <span class="code">$routeProvider</span> which is responsible for binding url address (ie. <span class="code">/home</span> suffix) with a proper behavior. In our case we want it to use a <span class="code">home.html</span> template for the view and <span class="code">HomeController</span> when there is <span class="code">/home</span> suffix present. In any other case we want our application to redirect to that address. Also we've defined a controller that is called <span class="code">HomeController</span> with no specific behavior.<br />
The last thing we need to do is to create a template for our home page. So create a file <span class="code">Calculator/app/view/home.html</span> and put this content inside it:<br />
<pre class="brush:html"><div>
<div>
This is home page:
</div>
<div>
You can go to <a href="#/calculator">Calculator</a>.
</div>
</div>
</pre>
As you can see it is very simple html code that will display a message and a link to the calculator page (the page we havn't created yet).<br />
<br />
<b>5. First try out of an application</b><br />
Now you can try to run this application. You can do this in two ways:<br />
<ul>
<li>Deploy it on http server and go to url of <span class="code">index.html</span> file</li>
<li>Open <span class="code">index.html</span> directly from your file system</li>
</ul>
For now, the latter one will be an easier solution ;)<br />
<br />
<u>Attention!</u><br />
Because of security reasons some browsers can block a javascript to access local files through AJAX request. This is the case at least for a Chrome and an Internet Explorer. I don't know the solution for an IE but if you want to test it in a Chrome you have to run the browser with <span class="code">--allow-file-access-from-files</span> parameter as stated on this <a href="http://stackoverflow.com/questions/18053547/angularjs-failed-to-load-resource-origin-null-is-not-allowed-by-access-control">stackoverflow question</a>. It should not be the case if you deploy your application in http server.<br />
For a Firefox browser our application should work flawlessly.<br />
<br />
<b>6. Add calculator page</b><br />
To add a calculator to our application first we need to create an html template that will define ours calculator's view. We'll put it inside <span class="code">Calculator/app/view/calculator.html</span> file:<br />
<pre class="brush:html"><div class="calculator">
<div class="display">0</div>
<div class="digitsKeyboard">
<div class="digitKey">1</div>
<div class="digitKey">2</div>
<div class="digitKey">3</div>
<div class="digitKey">4</div>
<div class="digitKey">5</div>
<div class="digitKey">6</div>
<div class="digitKey">7</div>
<div class="digitKey">8</div>
<div class="digitKey">9</div>
<div class="digitKey">0</div>
<div class="equalSignKey">=</div>
</div>
<div class="operationsKeyboard">
<div class="operationKey">/</div>
<div class="operationKey">*</div>
<div class="operationKey">+</div>
<div class="operationKey">-</div>
</div>
</div>
</pre>
So basically in our calculator we'll have a display, a keyboard with digit keys and an equal sign key, and a keyboard with possible mathematical operations. We also have to provide a stylesheet for our calculator so it can look properly. So put this CSS inside <span class="code">Calculator/app/style/default.css</span> file:<br />
<div style="height: 200px; overflow: auto;">
<pre class="brush:css">.calculator {
display: inline-block;
background-color: black;
height: 240px;
width: 208px;
overflow: hidden;
}
.display {
background-color: #DDDDDD;
border: 1px solid black;
font-family: monospace;
font-size: 27px;
font-weight: bold;
height: 30px;
padding: 0 4px;
text-align: right;
width: 198px;
overflow: hidden;
}
.digitsKeyboard {
overflow: hidden;
width: 156px;
float: left;
}
.digitKey {
width: 50px;
height:50px;
background-color: #AABBCC;
}
.equalSignKey {
width: 102px;
height: 50px;
background-color: #AACCBB;
}
.operationsKeyboard {
overflow: hidden;
width: 52px;
height:208px;
float: left;
}
.operationKey {
width: 50px;
height:50px;
background-color: #CCAABB;
}
.digitKey, .equalSignKey, .operationKey {
cursor: pointer;
font-family: monospace;
font-size: 33px;
border: 1px solid black;
line-height: 50px;
text-align: center;
float: left;
}
.digitKey:hover, .equalSignKey:hover, .operationKey:hover {
opacity: 0.8;
}
.digitKey:active, .equalSignKey:active, .operationKey:active {
opacity: 0.7;
}
</pre>
</div>
<br />
The last thing we have to do is to map a <span class="code">/calculator</span> address in our application. So we need to add a calculator page to <span class="code">$routeProvider</span> in exactly the same way we've defined a home page. So add this inside <span class="code">Calculator/app/js/app.js</span> file:<br />
<pre class="brush:javascript">$routeProvider.when("/calculator", {
templateUrl: "app/view/calculator.html",
controller: "CalculatorController"
}); </pre>
And also define new empty controller for Calculator:
<br />
<pre class="brush:javascript">app.controller("CalculatorController", function () {
});
</pre>
Now you can try to access calculator page in your browser. Just open the <span class="code">index.html</span> file and click on the calculator link. Then you should see a calculator on the screen.<br />
<br />
<b>7. Define calculator constants</b><br />
Now we'll add some juice to our <span class="code">CalculatorController</span>. To do that let's start by changing the factory function declaration for the controller:<br />
<pre class="brush:javascript">app.controller("CalculatorController", function ($scope) {
});</pre>
As you can see we've added a <span class="code">$scope</span> parameter to a function definition. It is an object where our application state is held. For our purpose you just need to know that you can define any field inside the $scope object and it will be accessible in other places in an application.<br />
<br />
So let us start with defining a keys of calculator with the required properties by putting this code inside controller's function.<br />
<pre class="brush:javascript">$scope.digitKeys = [
{label: "1", value: 1}, {label: "2", value: 2}, {label: "3", value: 3},
{label: "4", value: 4}, {label: "5", value: 5}, {label: "6", value: 6},
{label: "7", value: 7}, {label: "8", value: 8}, {label: "9", value: 9},
{label: "0", value: 0}
];</pre>
We'll use <span class="code">digitKeys</span> field to define an array of objects that represent every digit key in calculator keyboard. Every key has a <span class="code">label</span> property that will be displayed on given key and a value that will be used for computation.<br />
<pre class="brush:javascript">$scope.equalSignKey = {label: "="};</pre>
Here we only need a label for equal sign key. This code is placed here only to be consistent with other keys definitions. Other solution would be just to place equal sign directly inside html file.<br />
<pre class="brush:javascript">$scope.operationKeys = [
{label: "/", operation: function (a, b) {return a / b}},
{label: "*", operation: function (a, b) {return a * b}},
{label: "+", operation: function (a, b) {return a + b}},
{label: "-", operation: function (a, b) {return a - b}}
];</pre>
And the last set of keys are the mathematical operation keys. As you can see we put the operation function inside every object, so that will give us easy access to them, when we'll want to call it.<br />
<br />
<b>8. Make calculator rendered dynamically</b><br />
When we defined our buttons in CalculatorController we can use them to simplify our html template and generate buttons dynamically. Let's start with the easiest case of equal sign key:<br />
<pre class="brush:html"><div class="equalSignKey">
{{ equalSignKey.label }}
</div></pre>
In this case we used <span class="code">{{ }}</span> placeholder to tell AngularJS that we want to put <span class="code">equalSignKey.label</span> value in this place. Angular will look for that value within the <span class="code">$scope</span> object and it will bind it with a corresponding part of an html file.<br />
<br />
We'll do similar thing with digit buttons, but we'll use an AngularJS directive named <span class="code">ng-repeat</span> that allow us to iterate over array elements and repeat an html tag for every element in array:<br />
<pre class="brush:html"><div class="digitKey" ng-repeat="key in digitKeys">
{{ key.label }}
</div></pre>
In this case AngularJS iterates over <span class="code">$scope.digitKeys</span> array and puts current element inside a key variable. That way we don't have to repeat html code for every key.<br />
<br />
We'll do exactly the same operation for operations keys:<br />
<pre class="brush:html"><div class="operationKey" ng-repeat="key in operationKeys">
{{ key.label }}
</div></pre>
So the resulting <span class="code">calculator.html</span> file should look like that:<br />
<pre class="brush:html"><div class="calculator">
<div class="display">0</div>
<div class="keyboard">
<div class="digitKey" ng-repeat="key in digitKeys">
{{ key.label }}
</div>
<div class="equalSignKey" ng-click="compute()">
{{ equalSignKey.label }}
</div>
</div>
<div class="operations">
<div class="operationKey" ng-repeat="key in operationKeys">
{{ key.label }}
</div>
</div>
</div></pre>
<br />
Now you can test the application to check if it still displays properly.<br />
<br />
<b>9. Add calculator behavior</b><br />
Now we want to make our calculator to behave as real calculator so we'll need to define a variables that will hold a current calculator state. We'll need 5 of those:<br />
<ul>
<li><span class="code">displayValue</span> - current value displayed on calculator screen</li>
<li><span class="code">valueA</span> - first (left) value that will be used for computation</li>
<li><span class="code">valueB</span> - second (right) value that will be used for computation</li>
<li><span class="code">selectedOperation</span> - which mathematical operation was chosen by the user </li>
<li><span class="code">clearValue</span> - should value displayed on screen be cleared after new digit pressed?</li>
</ul>
So let's define them and initialize them with default values. Add this code inside a <span class="code">CalculatorController</span>:<br />
<pre class="brush:javascript">$scope.displayValue = 0;
$scope.valueA = 0;
$scope.valueB = 0;
$scope.selectedOperation = null;
$scope.clearValue = true;</pre>
<br />
<b>10. Bind displayValue variable with view</b><br />
As we now have a variable <span class="code">displayValue</span>, we would like it to be displayed in our calculator. And in this place best of AngularJS comes in. To do that just put a <span class="code">{{ displayValue }}</span> placeholder inside a <span class="code">calculator.html</span> changing <span class="code"><div class="display">0</div></span> so it will look like this:<br />
<pre class="brush:html"><div class="display">{{ displayValue }}</div></pre>
That's all!<br />
Now AngularJS we'll do it's tricks to ensure that value displayed on page is equal to the value of the variable. If you change the variable value, then automatically the value displayed on the page will be changed to be the same as the variable.<br />
And that's not all, imagine if display would be editable input field, then when user would input some value inside it, then automatically the value of the binded variable would be changed as well.
So it works both ways and it's hugely simplifies development of an application.<br />
<br />
<b>11. Add behavior to calculator's buttons</b><br />
We have all parts of our calculator in place, so now it's time to add a behavior to it. So the main idea is to bind a buttons with correct behavior. To do this with AngularJS we'll use <span class="code">ng-click</span> directive. So let's change the <span class="code">calculator.html</span> template:<br />
<br />
<pre class="brush:html"><div class="calculator">
<div class="display">{{ displayValue }}</div>
<div class="digitsKeyboard">
<div class="digitKey" ng-repeat="key in digitKeys"
ng-click="digitClicked(key.value)">
{{ key.label }}
</div>
<div class="equalSignKey" ng-click="compute()">
{{ equalSignKey.label }}
</div>
</div>
<div class="operationsKeyboard">
<div class="operationKey" ng-repeat="key in operationKeys"
ng-click="operationClicked(key.operation)">
{{ key.label }}
</div>
</div>
</div></pre>
<br />
In this part of code we've added three <span class="code">ng-click</span> directives that contains simple JavaScript function calls:<br />
<ul>
<li>For a digit key: <span class="code">ng-click="digitClicked(key.value)"</span></li>
<li>For an operation key: <span class="code">operationClicked(key.operation)"</span></li>
<li>For an equal sign key: <span class="code">ng-click="compute()"</span></li>
</ul>
As you can see we could pass a data defined earlier in AngularJS variables in the same way as we've passed them earlier into <span class="code">{{ }}</span> blocks. The main idea for those changes is to call a corresponding function after clicking on a button.<br />
<br />
Now let's define those three functions inside <span class="code">CalculatorController</span>. It is important to notice that those functions have to be defined as a part of <span class="code">$scope</span> object:<br />
<ul>
<li>Function to handle digit button click:</li>
</ul>
<div>
<pre class="brush:javascript">$scope.digitClicked = function (digit) {
if ($scope.clearValue) {
$scope.displayValue = digit;
$scope.clearValue = false;
} else {
$scope.displayValue = $scope.displayValue * 10 + digit;
}
$scope.valueB = $scope.displayValue
};</pre>
</div>
After user clicks a digit button, we want to update displayed value by replacing displayed value with digit, or by increasing displayed value by that digit (it's simple, just scale previous value by the factor of 10 and then add clicked digit). Also we need to remember displayed value as second value for the mathematical operation.<br />
<ul>
<li>Function to handle operation button click:</li>
</ul>
<pre class="brush:javascript">$scope.operationClicked = function (operation) {
$scope.selectedOperation = operation;
$scope.valueA = $scope.displayValue;
$scope.valueB = $scope.displayValue;
$scope.clearValue = true;
};</pre>
When user clicks operation button we simply want to remember clicked operation, and assign currently displayed value as both values for mathematical operation. Also we need to set <span class="code">clearValue</span> flag, so when user clicks on digit button, the displayed value will be replaced by selected digit, not increased by it.
<br />
<ul>
<li>Function to handle equal sign button click:</li>
</ul>
<div>
<pre class="brush:javascript">$scope.compute = function () {
if($scope.selectedOperation != null) {
$scope.displayValue = Math.floor(
$scope.selectedOperation($scope.valueA, $scope.valueB));
$scope.clearValue = true;
$scope.valueA = $scope.displayValue;
}
}</pre>
The last function main responsibility is to calculate a result if an operation is selected (assigned to <span class="code">$scope.selectedOperation</span> field). It is done by simple call of remembered function. Also <span class="code">clearValue</span> flag is set and a computation result is remembered as first value of new operation.
</div>
<br />
<b>Summary</b><br />
This is all, now we have a fully functional calculator for internet browsers.<br />
As you can see AngularJS is an easy to use (at least for an online calculator) and quite powerful framework for JavaScript application development. It brings a new way to bind an application state with html based view layer, that greatly simplifies creation of JavaScript applications.<br />
<br />
The complete source code of this calculator is available at <a href="https://github.com/marpiec/AngularJSCalculator" target="_blank">github.com</a>.Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com14tag:blogger.com,1999:blog-2508340156146834234.post-40054187067355753502013-07-30T17:30:00.000+02:002013-08-01T11:10:24.442+02:00Faking System Clock with metaprogramming pattern in ScalaTomasz Nurkiewicz has written an article <a href="http://nurkiewicz.blogspot.com/2013/07/fake-system-clock-pattern-in-scala-with.html">Fake system clock pattern in Scala with implicit parameters</a> on his blog. The problem he addresses is that sometimes you require to test your business code against a given date, ie. your system should behave differently on the weekends.<br />
<br />
Obviously this will be impossible (or very hard) to handle if a programmers would directly use <span class="code">new Date()</span>, <span class="code">System.curentTimeMills()</span> or some similar construct provided by JVM. So Tomasz proposed an application wide <span class="code">Clock</span> trait that is kind of proxy to <a href="http://joda-time.sourceforge.net/">Joda Time</a>:<br />
<pre class="brush:scala">import org.joda.time.{DateTime, Instant}
trait Clock {
def now(): Instant
def dateNow(): DateTime
}</pre>
With the default implementation:<br />
<pre class="brush:scala">import org.joda.time.{Instant, DateTime}
object SystemClock extends Clock {
def now() = Instant.now()
def dateNow() = DateTime.now()
}
</pre>
I like this approach very much, because it provides a clean way to grab the current date, and it decouples us from specific Time library (Joda Time in this case). It is quite obvious how programmer should grab the current system time:<br />
<pre class="brush:scala">val currentTime = SystemClock.now
</pre>
Now let's back to initial problem, how to pass a specific time if we want to test our code? I propose to use the method I've described in <a href="http://mpieciukiewicz.blogspot.com/2013/07/metaprogramming-in-scala-with-higher.html">Metaprogramming in Scala with higher order functions</a> article.<br />
<br />
<br />
<b>First Step - Create companion object for clock</b><br />
Let's create a companion object <span class="code">Clock</span> that will provide, to a programmer, a kind of proxy to default clock implementation, so the programmer won't have to use <span class="code">SystemClock</span> directly:<br />
<pre class="brush:scala">object Clock {
private val clockInstance: Clock = SystemClock
def now() = clockInstance.now()
def dateNow() = clockInstance.dateNow()
}
</pre>
And now the usage of our <span class="code">Clock</span> has been simplified to this:
<br />
<pre class="brush:scala">val currentTime = Clock.now
</pre>
<br />
<b>Second step - Allow to change clock instance</b><br />
To solve our problem of testing against specific time we would like to allow programmer to change the clock instance that is used in <span class="code">Clock</span> companion object:<br />
<pre class="brush:scala">object Clock {
private var clockInstance: Clock = SystemClock
def now() = clockInstance.now()
def dateNow() = clockInstance.dateNow()
def changeClock(newClock:Clock) {
clockInstance = newClock
}
}
</pre>
Now the programmer can switch the instance of <span class="code">Clock</span> in the following way:
<br />
<pre class="brush:scala">
//alternative Clock implementation
class FakeClock(fixed: DateTime) extends Clock {
def now() = fixed.toInstant
def dateNow() = fixed
}
//change of clock that is used
Clock.changeClock(new FakeClock(new DateTime(2013, 7, 15, 0, 0, DateTimeZone.UTC)))
//code to be tested
...
val currentTime = Clock.now
...
//revert the clock change
Clock.changeClock(SystemClock)
</pre>
As you can see we allowed a programmer to force the <span class="code">Clock</span> object to return specific date instead of current system date. To use it in tests a programmer only have to call <span class="code">Clock.changeClock</span> function before tested code, and a tested code will "see" that current time is different.
<b> </b><br />
<br />
<br />
<b>Final step - change Clock instance by declaration</b><br />
In the final step we'll change our <span class="code">Clock.changeClock</span> function to higher order function so it's usage will be much nicer and it will allow to change clock instance temporarily only.<br />
<pre class="brush:scala">object Clock {
private var clockInstance: Clock = SystemClock
def now() = clockInstance.now()
def dateNow() = clockInstance.dateNow()
def changeClock(newClock:Clock)(codeBlock: => Unit) {
val oldClock = Clock.clockInstance
clockInstance = newClock
codeBlock
clockInstance = oldClock
}
}
</pre>
So now to change a clock instance a programmer could write something like this:
<br />
<pre class="brush:scala">import Clock.changeClock
// changing clock instance only for the given block of code
changeClock(new FakeClock(new DateTime(2012, 2, 2, 0, 0, DateTimeZone.UTC))) {
...
val currentTime = Clock.now
...
}
</pre>
<br />
<b>Usage example</b><br />
This is a little more complicated usage example, but it's a nice way to demonstrate how easy the change of clock is now:
<br />
<pre class="brush:scala">object ClockTest {
import Clock.changeClock
def testedMethod() {
println(Clock.dateNow())
}
def main(args: Array[String]) {
testedMethod()
changeClock(new FakeClock(new DateTime(2013, 7, 15, 0, 0, DateTimeZone.UTC))) {
testedMethod()
changeClock(new FakeClock(new DateTime(2012, 2, 2, 0, 0, DateTimeZone.UTC))) {
testedMethod()
}
testedMethod()
}
testedMethod()
}
}
</pre>
And the output of this simple program would be this (let's assume that now is 30th of July 2013):
<br />
<pre class="brush:plain">2013-07-30T19:02:38.501+02:00
2013-07-15T00:00:00.000Z
2012-02-02T00:00:00.000Z
2013-07-15T00:00:00.000Z
2013-07-30T19:02:38.594+02:00
</pre>
As you can see this worked very well even for nested blocks of <span class="code">changeClock</span>.<br />
<br />
<br />
The complete implementation is available on <a href="https://gist.github.com/marpiec/5a4a11ce12a9bf81aff9">gist.github.com</a>.
<br />
<br />
<br />
Previous article: <a href="http://mpieciukiewicz.blogspot.com/2013/07/metaprogramming-in-scala-with-higher.html">Metaprogramming in Scala with higher order functions</a>Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com2tag:blogger.com,1999:blog-2508340156146834234.post-74282915671932368402013-07-21T20:00:00.000+02:002013-07-22T14:59:32.732+02:00Metaprogramming in Scala with higher order functionsScala is very flexible programming language. In this article I would like to propose some usage pattern for higher order functions. This pattern could help us to create a convenient way to handle multiple situations such as:<br />
- monitoring code performance<br />
- exception logging<br />
- executing commands in separate Thread<br />
- transaction handling<br />
- try with resource<br />
- environment changing<br />
<br />
Below I'm giving an usage examples and sample implementations for every of this cases.<br />
<br />
<b>Higher order function as meta-data</b>
<br />
Basically higher order function is a function that takes another function as a parameter. In the following example <span style="font-family: "Courier New",Courier,monospace;">calculate</span> is a higher order function that takes function <span style="font-family: "Courier New",Courier,monospace;">sum</span> as parameter :<br />
<pre class="brush:scala">scala> def sum(a:Int, b:Int) = a + b
scala> def calculate(a:Int, b:Int, operation: (Int, Int) => Int) = operation(a, b)
scala> calculate(5, 3, sum)
res1: Int = 8
</pre>
We will leverage the fact that we can pass a function that does not take any parameters, and does return nothing. That means that we'll simply pass a code block as a parameter to our function. This way we can do this:<br />
<pre class="brush:scala">//implementation
def doItTwice(codeBlock: => Unit) {
codeBlock
codeBlock
}
//usage with full syntax
doItTwice({
println("Hello!")
})
</pre>
and to do this even nicer we can skip parentheses in <span style="font-family: "Courier New",Courier,monospace;">doItTwice</span> call, so it will look this way:
<br />
<pre class="brush:scala">//proper usage with abbreviated syntax
doItTwice {
println("Hello!")
}
</pre>
the result is simple to predict:
<br />
<pre>Hello!
Hello!
</pre>
Doesn't it feel like we have extended language we are using? Now it's almost like adding simple metadata to our application.<br />
This feature can be leveraged to improve our programs, but before that we have to use another Scala feature called currying. It allows us to define multiple parameters lists for a function. Thanks to that, we will be able to pass another parameters to our higher function. Will take a closer look at this by introducing new loop implementation.<br />
<br />
<b>Simple loop</b>
<br />
Let's assume we want to define a simple way to loop our program for given number of times. And we want to do this as simple as possible. In example we would like to use it in this way:
<br />
<pre class="brush:scala">//sample usage
repeat(5) {
println("Cool loop")
}</pre>
Now we should be able to do this easily. This is the solution:
<br />
<pre class="brush:scala">//implementation
def repeat(times: Int)(codeBlock: => Unit) {
var i = 0
while (i < times) {
codeBlock
i += 1
}
}
</pre>
Thanks to the currying we could have passed <span style="font-family: "Courier New",Courier,monospace;">times</span> parameter and still be able to use abbreviated syntax when passing a code block. This is very simple usage of this pattern, but it uses every feature of Scala we need. Let's go to more serious problem.<br />
<br />
<b>Execution time logging</b>
<br />
Next case. Let's assume that we have some complex block of code, and we have to monitor it's execution time. So the nice way to do this would be like that:
<br />
<pre class="brush:scala">//sample usage
logExecutionTime("My complex operations") {
complexOperation1
complexOperation2
complexOperation3
}
</pre>
with the result looking like this:<br />
<br />
<span style="font-family: "Courier New", Courier, monospace;">Execution time of "My complex operations" was 125 mills</span><br />
<br />
To achieve this we can simply define <span style="font-family: "Courier New",Courier,monospace;">logExecutionTime</span> function this way:
<br />
<pre class="brush:scala">//implementation
def logExecutionTime(name: String)(codeBlock: => Unit) {
val start = System.currentTimeMillis()
codeBlock
val end = System.currentTimeMillis()
println("Execution time of \""+ name +"\" was "+ (end - start)+" mills")
}
</pre>
Btw. I have proposed a little more complex solution to measure time execution and described it in <a href="http://mpieciukiewicz.blogspot.dk/2013/07/neat-and-simple-way-to-measure-code.html">Neat and simple way to measure code performance in Scala</a> article.<br />
<br />
<b>Exception logging</b>
<br />
Now let's think about exceptions. There are many cases when you need just output the information about exception that occurred, but your application or maybe container of the application will handle that exception somewhere else. In the following example <span style="font-family: "Courier New",Courier,monospace;">logExceptions</span> will catch exception thrown from inner code block, write it out to a console, and than rethrow it, so the program can handle it:
<br />
<pre class="brush:scala">//sample usage
logExceptions {
operationThatMightCauseException
}</pre>
And this is simple implementation of it.
<br />
<pre class="brush:scala">//sample usage
def logExceptions(codeBlock: => Unit) {
try {
codeBlock
} catch {
case e:Exception => {
e.printStackTrace()
throw e
}
}
}
</pre>
I hope that now you've got feeling that this approach can be treated as metaprogramming, in this case <span style="font-family: "Courier New",Courier,monospace;">logException</span> can be treated as an annotation to a given code block. And, in my humble opinion, this is very nice. So let's go to further possible usages of this pattern.<br />
<br />
<b>Execute commands in separate Thread</b>
<br />
So maybe we would like to execute some commands separately to our main program thread, ie. to send a message to external server. It have to be executed in separate thread, so it wouldn't block our application because of network delays. So that might be the way we would like to program it:
<br />
<pre class="brush:scala">//possible usage
fireAndForget {
sendMessageViaHttp
}
</pre>
In the simplest solution we'll just create a new thread and run a given block of code in that thread:
<br />
<pre class="brush:scala">//implementation
def fireAndForget(codeBlock: => Unit) {
new Thread(new Runnable {
def run() {
codeBlock
}
}).start()
}
</pre>
<b>Transaction handling</b>
<br />
Now let's think about a common pattern in enterprise application - a transaction handling in database communication. In simplest situation you will to start a transaction, then execute your database commands and in the end you will commit the transaction or rollback it if something went wrong. We would like to help a programmer to do this multiple of times, so it would be nice if he could simply declare a transaction around the database commands. In the example I've assumed that our database commection is represented by simple trait with three methods to control a transaction:
<br />
<pre class="brush:scala">trait Connection {
def beginTransaction()
def commit()
def rollback()
}
</pre>
And now the programmer should be able to program like that:
<br />
<pre class="brush:scala">//possible usage
transaction(postgresConnection) {
insertRecords
updateOtherRecords
}
</pre>
The <span style="font-family: "Courier New",Courier,monospace;">transaction</span> is responsible for creating a transaction, executing commands and commiting or rollbacking transaction (on the exception occurrence). So it's implementation could look like that:
<br />
<pre class="brush:scala">//implementation
def doInTransaction(connection: Connection)(codeBlock: => Unit) {
connection.beginTransaction()
try {
codeBlock
connection.commit()
} catch {
case e:Exception => {
connection.rollback()
throw e
}
}
}
</pre>
<b>Try with resource or simple resource handling, ie. file reading</b><br />
Java 7 introduced <span style="font-family: "Courier New",Courier,monospace;">try with resources</span> statement that facilitated usage of external resources that have to be handled with care. In example they have to be properly closed to be reused. Before that resources handling was error prone because programmers often forgot to close the resource. So let's look how we could simplify it in Scala with the example of file reading. One of the easiest way to read file could be like that:
<br />
<pre class="brush:scala">//possible usage
readFile("file.txt") {reader =>
println("First line " + reader.readLine())
println("Second line " + reader.readLine())
println("Third line " + reader.readLine())
}
</pre>
And the <span style="font-family: "Courier New",Courier,monospace;">fileRead</span> implementation should properly open a file, and at the end close it properly. That could be implemented in this way:
<br />
<pre class="brush:scala">//implementation
def readFile(filename:String)(codeBlock: BufferedReader => Unit) {
val reader = new BufferedReader(new FileReader((filename)))
try {
codeBlock(reader)
} finally {
if(reader != null) {
reader.close()
}
}
}
</pre>
<b>Environment change</b><br />
For the last example let's assume that we have a parts of application that depend on some specific environment configuration. What if we have to change it temporarily to do something specific, for example for test purposes.<br />
So let's assume that our environment is accessible for us by the singleton object that holds String to String map:
<br />
<pre class="brush:scala">object Environment {
var environment = Map[String, String]()
}
</pre>
And for our example let's assume that our environment contains an entry that holds ip address for some name server: <span style="font-family: "Courier New",Courier,monospace;">"nameServer" -> "192.168.10.210"</span>. If we would like to alternate it temporarily to redirect request to localhost we would like to do something like that:
<br />
<pre class="brush:scala">//possible usage
println(Environment.environment("nameServer"))
changeEnvironment("nameServer" -> "127.0.0.1") {
println(Environment.environment("nameServer"))
}
println(Environment.environment("nameServer"))
</pre>
and that should give us that output:
<br />
<pre>192.168.10.210
127.0.0.1
192.168.10.210
</pre>
To achive this, one possible implementation of changeEnvironment could look like this:
<br />
<pre class="brush:scala">//implementation
def changeEnvironment(change: (String, String))(codeBlock: => Unit) {
val oldEnvironment = Environment.environment
Environment.environment = Environment.environment + change
codeBlock
Environment.environment = oldEnvironment
}
</pre>
The last example was inspired by Tomasz Nurkiewicz in his blog article <a href="http://nurkiewicz.blogspot.com/2013/07/fake-system-clock-pattern-in-scala-with.html">Fake system clock pattern in Scala with implicit parameters</a>. He proposed a solution for switching a Clock object, that is used within application, for the purpose of test that might require testing against given date. In the comment to that article I've proposed a <a href="https://gist.github.com/marpiec/5a4a11ce12a9bf81aff9">different solution</a>, than given by Tomasz, based on environment changing described above.<br />
<br />
<b>Conclusion</b><br />
I hope I've given you an idea that Scala can be used for metaprogramming with very simple constructs. When we'll use it properly, it can easily separate a business logic of an application from technical details of the implementation and this will greatly improve the way the source code can be read and managed.<br />
<br />
Previous article: <a href="http://mpieciukiewicz.blogspot.com/2013/07/neat-and-simple-way-to-measure-code.html">Neat and simple way to measure code performance in Scala</a> Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com1tag:blogger.com,1999:blog-2508340156146834234.post-61586117515580719632013-07-17T18:30:00.000+02:002013-07-22T14:58:41.438+02:00Neat and simple way to measure code performance in ScalaWhen we write a source code we often have to be aware of the performace of our solution. The easiest way to achieve this is by measuring duration of the execution by taking the system time before and after our block code, ie.<br />
<br />
<h2>
Where is the problem?</h2>
<pre class="brush:scala">val startTime = System.nanoTime
timeConsumingOperations()
val endTime = System.nanoTime
println("Operations took " + (endTime - startTime) + " nano seconds")
</pre>
This is most basic approach that can be used in the simplest cases. But what if we want to measure times of multiple different operations? This simple solution would become a little complex and harder to maintain, ie.
<br />
<pre class="brush:scala">val startTime = System.nanoTime
operation1()
val middleTime1 = System.nanoTime
operation2()
val middleTime2 = System.nanoTime
operation3()
val endTime = System.nanoTime
println("Operation1 took " + (middleTime1 - startTime) + " ns")
println("Operation2 took " + (middleTime2 - middleTime1) + " ns")
println("Operation3 took " + (endTime - middleTime2) + " ns")
</pre>
That code is becoming hard to read, maintain and, in my opinion the worst, it almost hides our business code.<br />
Now let's compilcate it even more by assuming that we need to run those operations multiple times (ie. in a for loop) and we need to measure their entire time of execution. We could try to do something like that:
<br />
<pre class="brush:scala">var operation1Duration = 0L
var operation2Duration = 0L
var operation3Duration = 0L
for(i <- 1 to 10) {
val startTime = System.nanoTime
operation1()
val middleTime1 = System.nanoTime
operation2()
val middleTime2 = System.nanoTime
operation3()
val endTime = System.nanoTime
operation1Duration += middleTime1 - startTime
operation2Duration += middleTime2 - middleTime1
operation3Duration += endTime - middleTime2
}
println("Operation1 took " + operation1Duration + " ns")
println("Operation2 took " + operation2Duration + " ns")
println("Operation3 took " + operation3Duration + " ns")
</pre>
And that is an ugly piece of code.<br />
<br />
<h2>
Solution Proposal</h2>
I propose a simple solution to that problem, a simple stopwatch. With this stopwatch we can refactor our third case into this:
<br />
<pre class="brush:scala">clearStopwatchResults()
for(i <- 1 to 10) {
stopwatch("First operation") {
operation1()
}
stopwatch("Second operation") {
operation2()
}
stopwatch("Third operation") {
operation3()
}
}
println(stopwatchResults)
</pre>
And the result of this measurement would be something like this:
<br />
<pre class="brush:plain">Third operation -> 48 ms 355 us 124 ns
First operation -> 136 ms 219 us 210 ns
Second operation -> 644 ms 69 us 657 ns
</pre>
Basic idea for that approach is to create a Stopwatch that can measure execution time of a block of code and store it under given identifier (ie. "First operation"). It will accumulate a time of execution for given identifier, and in the end it will write out the measurement results in a readable format.<br />
<br />
<h2>
Solution Implementation</h2>
So here is the implementation of my Stopwatch:
<br />
<pre class="brush:scala">import collection.mutable
import scala.StringBuilder
class Stopwatch {
private val measurements = mutable.HashMap[Any, Long]()
def clearResults() {
measurements.clear()
}
def apply(identifier:Any)(codeBlock: => Unit) {
val start = System.nanoTime
codeBlock
val end = System.nanoTime
val oldValue = measurements.getOrElse(identifier, 0L)
measurements += identifier -> (oldValue + end - start)
}
def results():String = {
ResultsFormatter.format(measurements)
}
}
</pre>
And there is also a companion object to provide default Stopwatch and convenient methods to use it.
<br />
<pre class="brush:scala">object Stopwatch {
private val defaultStopwatch = new Stopwatch
def stopwatch(identifier:Any)(codeBlock: => Unit) {
defaultStopwatch(identifier)(codeBlock)
}
def stopwatchResults() = defaultStopwatch.results()
def clearStopwatchResults() {
defaultStopwatch.clearResults()
}
}
</pre>
To complete the solution we also need a class to format our measurements results into well formatted String:
<br />
<pre class="brush:scala">object ResultsFormatter {
def format(measurements: mutable.HashMap[Any, Long]):String = {
val sb = new StringBuilder
measurements.foreach(appendResult(sb))
sb.toString()
}
private def appendResult(sb: StringBuilder)(result: (Any, Long)) {
val identifier = result._1
val value = result._2
sb.append(identifier).append(" -> ")
appendIfNotZero(sb, extractSeconds(value), "s")
appendIfNotZero(sb, extractMills(value), "ms")
appendIfNotZero(sb, extractMicro(value), "us")
appendIfNotZero(sb, extractNano(value), "ns")
sb.append('\n')
}
private def extractNano(value: Long): Long = value % 1000
private def extractMicro(value: Long) = (value / 1000) % 1000
private def extractMills(value: Long) = (value / 1000000) % 1000
private def extractSeconds(value: Long) = value / 1000000000
private def appendIfNotZero(sb:StringBuilder, value:Long, suffix:String) {
if (value > 0) {
sb.append(value).append(" ").append(suffix).append(" ")
}
}
}
</pre>
I have to mention that this is not thread safe implementation and one instance of Stopwatch should be use within single thread.<br />
<br />
<h2>
Implementation description</h2>
<br />
<b><span style="font-family: courier; font-size: 17px;">Stopwatch</span> class</b><br />
The main point of the implementation is a Map called <span style="color: black; font-family: "Courier New",Courier,monospace;">measurements</span>. It is a mutable HashMap, for higher performance and it is used to map identifier into execution duration.<br />
<br />
The <span style="color: black; font-family: "Courier New",Courier,monospace;">Stopwatch</span> class provides three public methods:<br />
- <span style="color: black; font-family: "Courier New",Courier,monospace;"><b>apply(identifier:Any)(codeBlock: => Unit)</b></span> - This method takes an identifier and a code block that will be measured. It simply calls the codeBlock, calculates how long it took to execute it and in the end it accumulates the result in the measurements map.<br />
I think the method declaration requires a little more explanation. First it uses a feature called <a href="http://www.scala-lang.org/node/135">currying</a>. For our purpose we can assume that it allows us to define function parameters in multiple parentheses pairs (of course currying is much more and it can be used for other purposes, but this is topic for other aricle) instead of one. Also <span style="font-family: Courier New, Courier, monospace;">codeBlock: => Unit</span> part defines a function that takes no parameters, or just a code block surrounded by curly brackets { ... }.<br />
Thanks to that we could call apply method in this way (full syntax):<br />
<pre class="brush:scala">apply("some computation")({complexComputation()})
</pre>
or we can omit latter parentheses and write simply:
<br />
<pre class="brush:scala">apply("some computation"){complexComputation()}
</pre>
We wouldn't be able to do this, in such a nice way, if we would use only one parameters block, ie. <span style="font-family: Courier New, Courier, monospace;">apply(identifier:Any, codeBlock: => Unit)</span><br />
- <span style="color: black; font-family: "Courier New",Courier,monospace;"><b>clearResults()</b></span> - This method simply clears the measurements map to allow new measurements to be performed.<br />
- <span style="color: black; font-family: "Courier New",Courier,monospace;"><b>result()</b></span> - Returns a well formatted String representation of measurements result.<br />
<br />
As you can see, the core imlpementation of this solution is very clear and simple. Also this is very easy to add new functionalities to this class.
<br />
<br />
<br />
<b><span style="font-family: courier; font-size: 17px;">Stopwatch</span> companion object</b><br />
This companion object is used to increase the ease of using Stopwatch class. It provides default instance of Stopwatch and provides a convenient methods to use it. To use those methods we have to import this object into current namespace by:
<br />
<pre>import Stopwatch._
</pre>
This import statement will include all methods from Stopwatch object into current namespace, so we can call it directly, without passing Stopwatch name.
<br />
<br />
<br />
<b><span style="font-family: courier; font-size: 17px;">ResultsFormatter</span> class</b><br />
This class is simpy used to create a well formatted String from Map containing measurement results.<br />
<br />
<h2>
Conclusion</h2>
I think that this approach to code performance measurement is well suited for many use cases, when you don't want to use external libraries for microbenchmarking or use of external profilers. Scala is very flexible language that allows us to create simple and very elegant solution for that kind of problems.<br />
Of course my Stopwatch is very simple, and I can imagine many ways to improve it. In example it can be rebuilt to be thread safe, or internal map can be replaced by simple array to increase its performance. But I also think that this is a good point to start for most of the projects.<br />
<br />
Next article: <a href="http://mpieciukiewicz.blogspot.com/2013/07/metaprogramming-in-scala-with-higher.html">Metaprogramming in Scala with higher order functions</a><br />
Previous article: <a href="http://mpieciukiewicz.blogspot.com/2013/07/scala-parentheses-and-curly-brackets-in.html">Scala - Parentheses and Curly Brackets in Anonymous Functions</a>Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com34tag:blogger.com,1999:blog-2508340156146834234.post-14016311388603516062013-07-15T19:30:00.000+02:002013-07-22T14:56:50.816+02:00Scala - Parentheses and Curly Brackets in Anonymous FunctionsScala gives a lot of flexibility in defining anonymous functions, and that might lead to a confussion. Jacek Laskowski asked an interesting question on <a href="http://stackoverflow.com/questions/17639922/scala-compiler-says-error-identifier-expected-but-integer-literal-found-for/17649395">stackoverflow.com</a> regarding of calling a map method on collection. To summarize it (given a list lst):<br />
<br />
Why<span style="color: black; font-family: "Courier New",Courier,monospace;"> lst map {c:Char => 1} </span>and<span style="color: black; font-family: "Courier New",Courier,monospace;"> lst map ((c:Char) => 1) </span>work fine, but<span style="color: black; font-family: "Courier New",Courier,monospace;"> lst map (c:Char => 1) </span>gives us compilation error:<br />
<br />
<span style="color: black; font-family: "Courier New",Courier,monospace;"> error: identifier expected but integer literal found.<br /> lst map (c:Char => 1)</span><br />
<br />
To answer this question we should look into <a href="http://www.scala-lang.org/docu/files/ScalaReference.pdf" target="_blank">Scala Language Specification</a>, into part<span style="color: black; font-family: "Courier New",Courier,monospace;"> 6.23 Anonymous Functions</span>.There is a description how anonymous function can be defined:<br />
<br />
<div style="background-color: #f5f9fd; border: 1px solid #DDDDDD; padding: 1px 10px;">
<pre>Expr ::= (Bindings | [‘implicit’] id | ‘_’) ‘=>’ Expr
ResultExpr ::= (Bindings | ([‘implicit’] id | ‘_’) ‘:’ CompoundType) ‘=>’ Block
Bindings ::= ‘(’ Binding {‘,’ Binding} ‘)’
Binding ::= (id | ‘_’) [‘:’ Type]
</pre>
</div>
<br />
As you can see Bindings requires to be placed inside two surrounding parentheses. So if anonymous function defines the type of the parameter, as in our example, it has to be defined in this way:<br />
<pre class="brush:scala">(c:Char) => 1
</pre>
And the call to map should look like that:
<br />
<pre class="brush:scala">lst map((c:Char) => 1)
</pre>
Also in the same part of reference documentation we can find:<br />
<br />
<div style="background-color: #f5f9fd; border: 1px solid #DDDDDD; padding: 10px 10px;">
If an anonymous function <span style="color: black; font-family: "Courier New",Courier,monospace;">(x: T) => e</span> with a single typed parameter appears as the result expression of a block, it can be abbreviated to <span style="color: black; font-family: "Courier New",Courier,monospace;">x: T => e</span>. </div>
<br />
So if anonymous function is defied as last expression in a code block, and has exacly one parameter, you can use abbreviated syntax, without parenthesis around <span style="color: black; font-family: "Courier New",Courier,monospace;">c:Char</span>, to define anonymous function. So, in our example, we can write <span style="color: black; font-family: "Courier New",Courier,monospace;">c:Char => 1</span>, but only when we place it inside a code block <span style="color: black; font-family: "Courier New",Courier,monospace;">{c:Char => 1}</span>. And we can call map function this way:<br />
<pre class="brush:scala">lst map({c:Char => 1})
</pre>
or with abbreviated syntax without parenthesis:<br />
<pre class="brush:scala">lst map {c:Char => 1}
</pre>
And that explains main question why <span style="color: black; font-family: "Courier New",Courier,monospace;">lst map {c:Char => 1}</span> is legal, and <span style="color: black; font-family: "Courier New",Courier,monospace;">lst map (c:Char => 1) </span>is not.<br />
<br />
This is summarized in changelog at the end of the documentation (Changes in Version 2.1.7 (19-Jul-2006)):<br />
<br />
<div style="background-color: #f5f9fd; border: 1px solid #DDDDDD; padding: 10px 10px;">
Closure Syntax<br />
<br />
The syntax of closures has been slightly restricted (§6.23). The form<br />
<br />
x: T => E<br />
<br />
is valid only when enclosed in braces, i.e. { x: T => E }. The
following is illegal, because it might be read as the value x typed with
the type T => E:<br />
<br />
val f = x: T => E<br />
<br />
Legal alternatives are:<br />
<br />
val f = { x: T => E }<br />
val f = (x: T) => E</div>
<br />
<br />
<b>Another way to specify anonymous functions:</b><br />
<br />
If we look closer at specification we can see that it allows us to use another way to define anonymous function:<br />
<br />
<div style="background-color: #f5f9fd; border: 1px solid #DDDDDD; padding: 1px 10px;">
<pre>Expr ::= (Bindings | [‘implicit’] id | ‘_’) ‘=>’ Expr
</pre>
</div>
<br />
We can see that we can define your anonymous function without parameter binding in those two ways (if we don't need to specify argument type):<br />
<pre class="brush:scala">lst map (c => 1)
// or
lst map (_ => 1)
</pre>
I hope that this article clarified how we can declare anonymous functions and it shouldn't cause a confusion any more.<br />
<br />
Next article: <a href="http://mpieciukiewicz.blogspot.com/2013/07/neat-and-simple-way-to-measure-code.html">Neat and simple way to measure code performance in Scala</a><br />
Previous article: <a href="http://mpieciukiewicz.blogspot.com/2013/07/named-and-default-parameters.html">Named and Default parameters inconsistency in Scala</a>Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com3tag:blogger.com,1999:blog-2508340156146834234.post-1307496438010535222013-07-08T18:30:00.000+02:002013-07-22T14:54:48.157+02:00Named and Default parameters inconsistency in ScalaWhile I was reading <a href="http://www.amazon.com/Scala-Depth-Joshua-D-Suereth/dp/1935182706">Scala in Depth</a> I've run into an interesting problem related to named method parameters and class inheritance. What will happen, when in child class we'll override method and we will change parameters names in this method? How will it change the way that this method should be called? And what about default parameters, will those behave simillary to named parameters? It turns out that they will not behave the same. For this reason <b>inheritance of default parameters may result in difficult to detect errors in your application behavior</b>.<br />
<br />
I will start by summarizing ways that Scala gives us to pass parameters to method calls. How we can use named and default parameters which were introduced in Scala 2.8.<br />
<br />
<b>Positional parameters (likewise in Java).</b><br />
Usually parameters are passed to function by passing multiple parameters in the order they have been declared in a function declaration. ie.:<br />
<br />
<pre class="brush:scala">scala> def process(x1:Int, x2:Int) = x1 * 2 + x2
scala> process(2, 5)
res0: Int = 9
</pre>
<br />
<b>Named parameters.</b><br />
If we need to change the order of parameters in function call, or we want to improve readability of function call, or we want to pass only some of the parameters (if function allows that), we can use named parameters. ie.:<br />
<br />
<pre class="brush:scala">scala> def process(x1:Int, x2:Int) = x1 * 2 + x2
// passing parameters without change in ordering
scala> process(x1 = 2, x2 = 5)
res1: Int = 9
// passing parameters with change in ordering
scala> process(x2 = 5, x1 = 2)
res2: Int = 9
</pre>
<br />
It is also possible to call a function by combining positional parameters with named parameters. But in this case it is required that named parameters (if we change ordering of passed parameters) should be passed after positional parameters. ie.:<br />
<br />
<pre class="brush:scala">// some parameters with name
scala> process(2, x2 = 5)
res3: Int = 9
// first parameter named, BUT ordering was not changed
scala> process(x1 = 2, 5)
res4: Int = 9
// first parameter named, AND ordering was changed
scala> process(x2 = 5, 2)
<console>:9: error: positional after named argument.
process(x2 = 5, 2)
^
</pre>
<br />
<b>Default parameters.</b><br />
If we want allow a user of our function, not to pass all of the parameters, we can define, in a function declaration, a value that will be assigned to a parameter, if that parameter won't be passed in a function call. ie.:<br />
<br />
<pre class="brush:scala">scala> def sum(x1:Int, x2:Int, x3:Int = 0) = x1 + x2 + x3
scala> sum(1, 2, 3)
res5: Int = 6
scala> sum(1, 2)
res6: Int = 3
</pre>
<b><br /></b>
It is worth to notice, that default parameters don't have to be declared at the end of the parameters list. In that case the only way to call a function without passing theirs value is to use named parameters:<br />
<br />
<pre class="brush:scala">scala> def sum(x1:Int, x2:Int = 0, x3:Int) = x1 + x2 + x3
// only to parameters have been passed
scala> sum(1, 3)
<console>:9: error: not enough arguments for method sum: (x1: Int, x2: Int, x3: Int)Int.
Unspecified value parameter x3.
sum(1, 3)
// defined parameters were passed by name
scala> sum(x1 = 1, x3 = 3)
res7: Int = 4
</pre>
<br />
<br />
<b>Named parameters and class inheritance.</b><br />
Let's check how named parameters behave in the case of class inheritance and method overwriting. Let us look onto those example classes:<br />
<br />
<pre class="brush:scala">class Calculator {
def process(x1:Int, x2:Int) = x1 * 2 + x2
}
class BrokenCalculator extends Calculator {
override def process(a:Int, b:Int) = super.process(a, b) + 1
}
</pre>
<br />
It's important to notice that parameter names of method <span style="font-family: "Courier New",Courier,monospace;">process </span> has been channged from <span style="font-family: "Courier New",Courier,monospace;">x1</span> and <span style="font-family: "Courier New",Courier,monospace;">x2</span> to <span style="font-family: "Courier New",Courier,monospace;">a</span> and <span style="font-family: "Courier New",Courier,monospace;">b</span>.<br />
In the next step let's create instances of those classes, but in the case of <span style="font-family: "Courier New",Courier,monospace;">BrokenCalculator</span> instances we will assign them to references of different types:<br />
<pre class="brush:scala">val calculator = new Calculator
val brokenCalculator = new BrokenCalculator
val brokenCalculatorAsCalculator:Calculator = new BrokenCalculator
</pre>
<br />
Does assigning of <span style="font-family: "Courier New",Courier,monospace;">BrokenCalculator</span> to reference of type <span style="font-family: "Courier New",Courier,monospace;">Calculator</span> have an influance on method behavior? It turns out that it depends on the way how the parameters are passed:<br />
<br />
<pre class="brush:scala">scala> calculator.process(2, 5)
res8: Int = 9
scala> brokenCalculator.process(2, 5)
res9: Int = 10
scala> brokenCalculatorAsCalculator.process(2, 5)
res10: Int = 10
</pre>
<br />
In the case when we passed parameters as positional parameters all of the calls depends on the class of the instatiated object, therefore <span style="font-family: "Courier New",Courier,monospace;">brokenCalculator</span> and <span style="font-family: "Courier New",Courier,monospace;">brokenCalculatorAsCalculator</span> have behaved exacly the same.<br />
What if we want to use named parameters in method call?<br />
<br />
<pre class="brush:scala">scala> calculator.process(x2 = 5, x1 = 2)
res11: Int = 9
// parameter passed with the names from Calculator class
scala> brokenCalculator.sum(x2 = 5, x1 = 2)
<console>:11: error: value sum is not a member of BrokenCalculator
brokenCalculator.sum(x2 = 5, x1 = 2)
^
scala> brokenCalculator.process(b = 5, a = 2)
res12: Int = 10
// parameter passed with the names from BrokenCalculator class
scala> brokenCalculatorAsCalculator.sum(b = 5, a = 2)
<console>:11: error: value sum is not a member of Calculator
brokenCalculatorAsCalculator.sum(b = 5, a = 2)
^
scala> brokenCalculatorAsCalculator.process(x2 = 5, x1 = 2)
res13: Int = 10
</pre>
<br />
In this case, as you can see, it is important what reference type we are using. In our example objects <span style="font-family: "Courier New",Courier,monospace; font-size: small;">calculator</span> and <span style="font-family: "Courier New",Courier,monospace; font-size: small;">brokenCalculatorAsCalculator</span> required parameter names <span style="font-family: "Courier New",Courier,monospace;">x1</span> and <span style="font-family: "Courier New",Courier,monospace;">x2</span>. When we've tried to pass the names <span style="font-family: "Courier New",Courier,monospace;">a</span> and <span style="font-family: "Courier New",Courier,monospace;">b</span> we've got compilation error. Analogous in the case of <span style="font-family: "Courier New",Courier,monospace; font-size: small;">brokenCalculator</span> object we had to use names <span style="font-family: "Courier New",Courier,monospace;">a</span> and <span style="font-family: "Courier New",Courier,monospace;">b</span>, and when we've tried to use <span style="font-family: "Courier New",Courier,monospace;">x1</span> and <span style="font-family: "Courier New",Courier,monospace;">x2</span> we've also got compilation error.<br />
<br />
I think that for proper use of named parameters we have to assume that method's contract (the way how method can be called) is determined by the type of reference we use, and this allows compiler to verify the correctness of a method call.<br />
<br />
<b>Default parameters and class inheritance.</b><br />
Let's now change our classes, by defining a default parameter values. In the case of <span style="font-family: "Courier New",Courier,monospace;">BrokenCalculator</span> we'll change default value of <span style="font-family: "Courier New",Courier,monospace;">x2</span> parameter:<br />
<br />
<pre class="brush:scala">class Calculator {
def process(x1:Int, x2:Int = 0) = x1 * 2 + x2
}
class BrokenCalculator extends Calculator {
override def process(x1:Int, x2:Int = 1) = super.process(x1, x2)
}</pre>
Now let's check how method behavior depends on object type and reference type that it is assigned to:<br />
<br />
<pre class="brush:scala">scala> calculator.process(2)
res14: Int = 4 // default value 0
scala> brokenCalculator.process(2)
res15: Int = 5 // default value 1
scala> brokenCalculatorAsCalculator.process(2)
res16: Int = 5 // default value 1
</pre>
<br />
Those results are quite surprising (!), it seems that <b> if a child class changes the default value of a parameter than default values of parameters are indendent of the reference type</b>. During the call of <span style="font-family: "Courier New",Courier,monospace;">brokenCalculatorAsCalculator</span> parameter x2 had been assigned default value of 1, although the contract from <span style="font-family: "Courier New",Courier,monospace;">Calculator</span> class defined a default value of 0.
I think that this might cause problems, especially because IDE (in my case IntelliJ IDEA) shows to a developer that default value of <span style="font-family: "Courier New",Courier,monospace;">x2</span> in the call of <span style="font-family: "Courier New",Courier,monospace;">brokenCalculatorAsCalculator</span> is 0!<br />
<br />
One more case left. What if <span style="font-family: "Courier New",Courier,monospace;">BrokenCalculator</span> class won't declare any default value for <span style="font-family: "Courier New",Courier,monospace;">x2</span> parameter?<br />
<br />
<pre class="brush:scala">class BrokenCalculator extends Calculator {
override def process(x1:Int, x2:Int) = super.process(x1, x2)
}
</pre>
<br />
Can we now call the process method and pass it only 1 parameter?<br />
<br />
<pre class="brush:scala">scala> calculator.process(2)
res17: Int = 4
scala> brokenCalculator.process(2)
res18: Int = 4
scala> brokenCalculatorAsCalculator.process(2)
res19: Int = 4
</pre>
<br />
Yes we can! In this case <span style="font-family: "Courier New",Courier,monospace;">BrokenCalculator</span> has inherited default value of <span style="font-family: "Courier New",Courier,monospace;">x2</span> parameter, so <b>if child class doesn't define default parameter value, this value is inherited from parent class</b>.<br />
<br />
By this example you can see huge inconsistency in Scala design. What if some programmer will use the Calculator reference? <b>By looking at method defined in Calculator class he cannot know the default parameter value!</b> He has to look into concrete implementation of object he will have. But this is not always possible.<br />
<br />
So you have to remember that child class always inherits default values of parameters and it can always change them. And when you call the method you need to know the default values defined in concrete class, not only in reference type you are using.<br />
<br />
Next article: <a href="http://mpieciukiewicz.blogspot.com/2013/07/scala-parentheses-and-curly-brackets-in.html" target="">Scala - Parentheses and Curly Brackets in Anonymous Functions</a><br />
Previous article: <a href="http://mpieciukiewicz.blogspot.com/2013/07/list-sorting-in-scala.html">List sorting in Scala</a>Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com1tag:blogger.com,1999:blog-2508340156146834234.post-85542222499652548642013-07-01T12:46:00.003+02:002013-07-22T15:01:06.493+02:00List sorting in ScalaWhile I was watching <a href="http://jlaskowski.blogspot.com/">Jacek Laskowski (polish blog)</a> working on <a href="http://jlaskowski.blogspot.com/2013/06/mek-intelektualnych-kontynuacja.html">taskassin (also in polish)</a> and Scala programming I decided to summarize different approches to List sorting in Scala.<br />
In Jacek's problem we have to sort a list of tasks that are defined this way: (I simplified the class for the purpose of this article):<br />
<br />
<pre class="brush:scala">case class Task(label: String, due: DateTime)
</pre>
and we have defined list of objects:
<br />
<pre class="brush:scala">val tasks = List(Task("t1", DateTime.now),
Task("t2", DateTime.now + 2.hours),
Task("t3", DateTime.now - 2.hours))
</pre>
<i>Clarification: DateTime and .hours are from <a href="https://github.com/nscala-time/nscala-time">nscala-time</a> library that encapsulates JodaTime so it can be easily used in Scala.</i>
<br />
<br />
On a first glance we can see, that List class have three sorting methods:<br />
<b>.sorted</b> - to sort objects with defined Ordered trait<br />
<b>.sortBy</b> - to sort objects by selected field<br />
<b>.sortWith</b> - to sort objects by given ordering function<br />
Which one we will choose should depend on the business context of the application.
Let's look at some situations, in which we might need to sort tasks list, that might occur:<br />
<br />
<br />
<b>Case 1.</b> If sorting of tasks will always be done based on the value of due field, we can use object oriented programming and extend Task class with Ordered[Task] trait:
<br />
<pre class="brush:scala">case class Task(label: String, due: DateTime) extends Ordered[Task] {
def compare(other: Task): Int = this.due.compareTo(other.due)
}
</pre>
That way, the sorting of tasks list could be done this way:<br />
<pre class="brush:scala">val sortedTasks = tasks.sorted
</pre>
It's hard to imagine something easier from the programmer's, that uses the Task class, point of view.
<br />
<br />
<br />
<b>Case 2.1.</b> The different approach have to be made if we'll assume that tasks can be sorted in multiple ways, ie. by due time, or by a label. In this case we can pass a parameter to sorted method, that is an instantiation of Ordering[Task] with definition of the sorting order:
<br />
<pre class="brush:scala"> val sortedTasks = tasks.sorted(new Ordering[Task] {
def compare(a: Task, b: Task): Int = a.due.compareTo(b.due)
})
</pre>
<br />
also, you can use an easier way to create Ordering object:
<br />
<pre class="brush:scala">val sortedTasks = tasks.sorted(Ordering.fromLessThan[Task](_.due < _.due))
</pre>
<b>Case 2.2.</b>
Foregoing approach might seem not very elegant, but you can use it to create convenient way to sort tasks. To do so we can define ways of sorting in a companion object using Ordering[Task]:
<br />
<pre class="brush:scala">case class Task(label: String, due: DateTime)
object Task {
//full syntax of defining anonymous function
val orderingByDue = Ordering.fromLessThan((a:Task, b:Task) => a.due < b.due)
//short syntax of defining anonymous function
val orderingByLabel = Ordering.fromLessThan[Task](_.label < _.label)
}
</pre>
In this case you can sort tasks this way:<br />
<pre class="brush:scala">val tasksSortedByDue = tasks.sorted(Task.orderingByDue)
val tasksSortedByLabel = tasks.sorted(Task.orderingByLabel)
</pre>
<br />
<br />
<b>Case 3.</b> If, for some reason, we don't want to force programmer to use our way of sortnig, the sortBy method might be used. It takes, as a parameter, a field of the object that will be used to determine object sorting (by passing a getter function for that field):
<br />
<pre class="brush:scala">val sortedTasks = tasks.sortBy((t:Task) => t.due)
</pre>
Or you can use shorter and more readable syntax:
<br />
<pre class="brush:scala">val sortedTasks = tasks.sortBy(_.due)
</pre>
Additionally you can define more than one field that will be used for sorting (ie. sorting persons first by family name and, if those are the same, by the first name). You do that by passing a tuple of fields you are interested in:
<br />
<pre class="brush:scala">val sortedTasks = tasks.sortBy((x) => (x.label, x.due))
</pre>
In this case tasks will be sorted by name and, if some of them have the same name, by the due time.
<br />
<br />
<br />
<b>Case 4.1.</b> If solution from Case 3. is not enought for you (ie. sorting is more complicated), you can use sortBy method, that requires a function that compares two objects:<br />
<br />
<pre class="brush:scala">val sortedTasks = tasks.sortWith((a:Task, b:Task) => a.due < b.due)
val sortedTasks = tasks.sortWith(_.due < _.due)
</pre>
<b>Case 4.2.</b> It is important to notice, that in this case (alike in case 2.2.) it is possible to define those functions as an elements of a companion object:
<br />
<pre class="brush:scala">object Task {
val compareByDue = (a:Task, b:Task) => a.due < b.due
}
</pre>
And it can be used this way:
<br />
<pre class="brush:scala">val sortedTasks = tasks.sortWith(Task.compareByDue)
</pre>
<br />
<br />
<br />
<b>Sorting algorithms.</b>
While if was looking closely to the sorting in Scala, I've started to wonder about used sorting algoriths.<br />
<br />
Sorting of sequences (including List) by usage of sorted, sortBy or sortWith methods is made by the sorting algorithm provided by JDK, in particular by <b>java.util.Arrays.sort(...)</b> function. Scala rewrites the sequence to Array and it is passed to the sort function, than it is sorted in place. Than based on the sorted array a new sequence is created (based on a given sequence type, ie. List).<br />
<br />
Sorting method provided by JDK, used to sort array of objects, uses a modified mergesort algorithm, that guarantees sorting stability, and complexity O(n) = n log(n). Additionally, if input array is partially sorted, a performance of that algorithm is much greater.<br />
<br />
It is important to notice that java.util.Arrays.sort(...) function, in case of primitives' array, uses an optimized quicksort algorithm (as opposed to objects' array, because in case of primitives, it doesn't have to be stable), but to use it in Scala you have to call it manually.
<br />
<br />
<b>Summary.</b> My personal favor approaches to list sorting is presented in Case 1. and Case 2.2. bacause these allow us to define a business context, so the programmer won't have to discover it over and over (of course that will have more sense in more compilcated cases, than presented in taskassin). Additionally these approaches allow us to define properties of Task class, no it's behavior.
<br />
I know that solutions presented by me don't exhaust the topic of sorting in Scala, but I think that those are the most usefull ones.<br />
<br />
Next article: <a href="http://mpieciukiewicz.blogspot.com/2013/07/named-and-default-parameters.html">Named and Default parameters inconsistency in Scala</a> Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com2tag:blogger.com,1999:blog-2508340156146834234.post-23387649188241916552013-04-09T19:40:00.000+02:002013-04-10T10:49:39.138+02:00Scalatron Bot Mars imporved to reach 7.5 million energy points in benchmark!<p>
Hi, after playing a little more with Scalatron I've improved my bot in few areas:<br />
<br />
<b>Minibots count</b> - now my bot can spawn more mini bots and also my mini bots can spawn other mini bots so they have much more collective force<br />
<b>GoHome influence decreased, until the end of game</b> - mini bots are not so eager to capture master bot, but when game is near to end that force is getting much more important, so master bot could collect minibots energy<br />
<b>Master bot Hunger decreased and Explorer increased</b> - because there are a lot of mini bots in the game, the master bot don't have to look for the food, much more important is to wander with a high pace to collect spawned mini bots<br />
<br />
And the results are great. My bot reached about 7.5 million energy point in a benchmark, with 8 million in best round:<br />
<br />
<p style="text-align:center">
Benchmark average 20 rounds score:<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2IR1xCXSiUDla0_3X3H1ZBHW4cS0pK8JEBsx8i8Z8GcK8NOCZojZXKOBuNdvJ1wIpWvdyv5svVz-q2WiEClV8AAEylphk8wOITOhs0_5GDAARnkyydoOQ1YILPWaJ5ck6mjseaZNFZ5Po/s1600/mars-score-3.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2IR1xCXSiUDla0_3X3H1ZBHW4cS0pK8JEBsx8i8Z8GcK8NOCZojZXKOBuNdvJ1wIpWvdyv5svVz-q2WiEClV8AAEylphk8wOITOhs0_5GDAARnkyydoOQ1YILPWaJ5ck6mjseaZNFZ5Po/s320/mars-score-3.png" /></a>
</p>
<br />
<p style="text-align:center">
Maximum score:<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgajSnuq-OcKAW2KBxo0v37UzgBHN-A-B3VezQ2G4W0W4I2tTsUgoMokWoAitq7L2o0-f4s_xwroCO6d0lXHQrb8wvmxnmHHyILdQz4xKh5_v_zAoI7KT8XKCSEeS7hBU3ofZGKdXICBTPX/s1600/mars-score-2.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgajSnuq-OcKAW2KBxo0v37UzgBHN-A-B3VezQ2G4W0W4I2tTsUgoMokWoAitq7L2o0-f4s_xwroCO6d0lXHQrb8wvmxnmHHyILdQz4xKh5_v_zAoI7KT8XKCSEeS7hBU3ofZGKdXICBTPX/s320/mars-score-2.png" /></a>
</p>
<br />
Of course you can find updated source code on github's <a href="https://github.com/marpiec/ScalaBotMars">ScalaBotMars project</a>.
</p>Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com2tag:blogger.com,1999:blog-2508340156146834234.post-52921816210059602992013-04-03T18:32:00.000+02:002013-08-29T07:16:13.996+02:00Bot for Scalatron – great way to practice Scala<p>
Recently I was pointed to project <a href="http://scalatron.github.com/">Scalatron</a> which is a programming game, where bots are fighting in simple arena. The main goal is to capture as many energy points as possible, or at least more than your opponents.
</p>
<p style="text-align:center">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhi3w46ft_hyphenhyphenN0SU9rqIrAD6Y-ih454tzvVHfr1KGx3OnPXRhJgStrEGNkZMJGfyFYna7PL6mc_JcvnaRcDEumY9I0dyC6mWAWTtLz1-KRG6CMY8Rux3y13kPTpIsLEq6pL_eEzY9BMRCF_/s1600/screen.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhi3w46ft_hyphenhyphenN0SU9rqIrAD6Y-ih454tzvVHfr1KGx3OnPXRhJgStrEGNkZMJGfyFYna7PL6mc_JcvnaRcDEumY9I0dyC6mWAWTtLz1-KRG6CMY8Rux3y13kPTpIsLEq6pL_eEzY9BMRCF_/s320/screen.png" /></a>
</p>
<br />
<p>
Your goal, as a developer is to write a function in Scala programming language that controls your Bot. In brief your function will be called by the Scalatron server with passed parameters such as the map of area that is visible to your bot. And your function should return a commands that will be passed to your bot.
This is very simple mechanism that allows you to control your bot in easy way.
</p>
<p>
I consider the Scalatron project to be very enjoyable way to learn or practice the Scala language and trying to develop some interesting decisions algorithms.
</p>
<b>Scalatron Bot Mars to score 1 000 000 energy points!</b>
<p>
My idea to write a Scalatron bot was to equip it with a number of competitive senses and passions. One, to force bot to look after food, other to fear of the enemies and even a sense to fear of small rooms. Here are the list of those senses:
</p>
<p>
<b>Hunger</b> – looks after eatable beasts and plants<br />
<b>Fear</b> – tries to run away from opponents bots and hostile beasts<br />
<b>Explorer</b> – tries not to stay in one place for too long<br />
<b>CabinFever</b> – tries to avoid small rooms with, in some possibilities, no easy exit<br />
</p>
<p>
Those senses are used to calculate preferences for direction that bot should move. Most important characteristic of them is that they are exponentially decaying when the distance from interesting object is growing. That way bot will react mostly for the objects that are nearest to him.
</p>
<p>
Also, to be more efficient, my bot will simultaneously spawn child bots that might have one of two roles (that might be switched during lifetime of spawned bot):
</p>
<p>
<b>Missile</b> – to track down enemy bots or hostile beasts and blow up when they are near<br />
<b>Hunter</b> – to capture eatable beasts and plants<br />
</p>
<p>
Those bots, especially those in hunter role have additional senses besides those introduced earlier:
</p>
<p>
<b>Loner</b> – to avoid being in the same area with other friendly bots<br />
<b>GoHome</b> – to get back to my master bot to be absorbed by it<br />
</p>
<p>
Those are to summarise main ideas that I have come up with, that allowed me to develop a Mars bot that achieved more than 1 000 000 points in standard scalatron benchmark.
</p>
<p style="text-align:center">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgPXu15icJzJoDZQBFGHU5U0DgM6r58y0ceXxp0nnWn5J1RFHR0ggKKR7E6qwj5mij1Vh73BpU8QYru2BaXqG1N3kHA2P73E2UyUiSt904YPPMIfTAm1x9g_oR6zp706c5En8dMEtbIv4S/s1600/mars-score.png" imageanchor="1" ><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgPXu15icJzJoDZQBFGHU5U0DgM6r58y0ceXxp0nnWn5J1RFHR0ggKKR7E6qwj5mij1Vh73BpU8QYru2BaXqG1N3kHA2P73E2UyUiSt904YPPMIfTAm1x9g_oR6zp706c5En8dMEtbIv4S/s320/mars-score.png" /></a>
</p>
<p>
I you are interested in more details how I've implemented that bot, you can find my bot’s source code on github's <a href="https://github.com/marpiec/ScalaBotMars">ScalaBotMars project</a>.
</p>Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com1tag:blogger.com,1999:blog-2508340156146834234.post-4887042552011554822013-03-18T22:00:00.000+01:002013-03-18T16:57:02.566+01:00Spock testing framework introductionDuring <a href="http://2013.33degree.org/">33rd Degree</a> conference that took place in Warsaw in the middle on March 2013 I was listening to <a href="http://kensipe.blogspot.com/">Ken Sipe’s</a> speech entitled <a href="http://2013.33degree.org/talk/show/69">Mocking, Stubbing and Spying with Spock</a>. This has been very interesting and enjoyable introduction (at least for me) to <a href="http://spockframework.org/">Spock</a> testing and specification framework.<br />
<br />
What is Spock?<br />
<br />
Spock is a testing framework for Java Virtual Machine. It is developed in Groovy but is perfecly usable to test Java programs. Spock focuses on developing an easy to read and write specifications that extensively uses Groovy language features.<br />
<br />
So take a look how it might be used to test application, i.e. a simple calculator that is implemented with this Java code:<br />
<br />
<pre class="brush:java">
package pl.marpiec.calculator;
public class Calculator {
private int a;
private int b;
public Calculator(int a, int b) {
this.a = a;
this.b = b;
}
public int add() {
return a + b;
}
public int subtract() {
return a - b;
}
public int multiply() {
return a * b;
}
}
</pre>
<rb />
We want to test it’s ability to add, subtract and multiply features. To do it with spock we have to create groovy class that extends spock.lang.Specification class. And inside we can define a test method, i.e.:<br />
<br />
<pre class="brush:groovy">
package pl.marpiec.calculator
import spock.lang.Specification
import spock.lang.Unroll
class CalculatorSpec extends Specification {
def "Addition of two integers"() {
given:
def calculator = new Calculator(a, b)
expect:
calculator.add() == sum
where:
a | b || sum
-5 | 3 || -2
2 | 4 || 6
8 | -4 || 4
5 | 0 || 5
}
}
</pre>
Let’s have a closer look to this method, because it looks quite strange for java developers.<br />
<b>First.</b> Groovy allows you to name methods using quotes to define names containing spaces. That allows you to use more natural language, and easier to represent business requirements.<br />
<b>Second.</b> Our test method is divided into three blocks named “given”, “expect”, “where”. First is used to initialize our code, second one to define assertions (you might thing of it as it was assertTrue(calculator.add() == sum)), and the last one defines our test data in easy to read tabular form.<br />
<b>Third.</b> Specification defined in this way is in reality 4 tests with input data given in the where: block.<br />
<b>What is going on?</b>To perform a test Spock takes data defined in where: block and puts it into test code specified in given and expect code and then performs the tests.<br />
<br />
As you can see Spock’s Specification is very easy to read and define test data. What is also great in Spock is how it outputs the information about failed tests. In example let’s write another test method to test subtract feature of Calculator class, it is in purpose defined a little differently than the first one to enhance Spock abilities:
<pre class="brush:groovy">
def "Subtraction of two integers"() {
expect:
new Calculator(a, b).subtract() == difference
where:
a | b || difference
-5 | 3 || -8
2 | 4 || -2
8 | -4 || 12
5 | 0 || 5
}
</pre>
If we than broke the subtract method code to and fire the test:
<pre class="brush:java">
public int subtract() {
return b - a;
}
</pre>
The output of the test is this:
<pre class="brush:text">
Subtracting of two integers(pl.marpiec.calculator.CalculatorSpec) Time elapsed: 0.625 sec <<< FAILURE!
org.spockframework.runtime.SpockComparisonFailure: Condition not satisfied:
new Calculator(a, b).subtract() == difference
| | | | | |
| -5 3 8 | -8
| false
pl.marpiec.calculator.Calculator@1c54796
at pl.marpiec.calculator.CalculatorSpec.Subtracting of two integers(CalculatorSpec.groovy:25)
</pre>
<br />
This is realy beutiful test output, isn’t it? Spock writes out every importany value used in assertion, so no debugging is required to check what went wrong.<br />
<br />
And the last feature example will help us to determine what input data was used to performe a test. That is especially helpful if we would’nt use the input data directly in the expect: block, as it was done in method that tested addition. In spock we could use parameters in method name(!) using #variableName convention. Lets create test method for multiply feature of Calculator:
<pre class="brush:groovy">
@Unroll
def "Multiplication of two integers (#a * #b = #product)"() {
given:
def calculator = new Calculator(a, b)
expect:
calculator.multiply() == product
where:
a | b || product
-5 | 3 || -15
2 | 4 || 8
8 | -4 || -32
5 | 0 || 0
}
</pre>
<br />
The important parts are @Unroll adnotation that allows us (among the others) to use parameters in method name, and expression “#a * #b = #product” inside the method name. Now when error fails we’ll see the following output:<br />
<pre class="brush:text">
Multiplying of two integers (-5 * 3 = -15)(pl.marpiec.calculator.CalculatorSpec) Time elapsed: 0.11 sec <<< FAILURE!
org.spockframework.runtime.SpockComparisonFailure: Condition not satisfied:
calculator.multiply() == product
| | | |
| -2 | -15
| false
pl.marpiec.calculator.Calculator@147358f
at pl.marpiec.calculator.CalculatorSpec.Multiplying of two integers (#a * #b = #product)(CalculatorSpec.groovy:41)
</pre>
<br />
<br />
Those of course aren’t all the Spock’s features, but I hope this is a good example to start with Spock. I’m for sure will digg into other Spock's features.<br />
<br />
The example code with maven build definition can be found on github here <a href="https://github.com/marpiec/HelloSpock">HelloSpock</a>.Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com0tag:blogger.com,1999:blog-2508340156146834234.post-79800364832411411852013-03-05T09:57:00.000+01:002013-08-20T17:30:52.118+02:00Maven configuration example for ScalaWhen you start to learn Scala, you might like to use tools that you are familiar to. I.e. you would like to use well known Maven to build your project and manage dependencies. In this post I’ll show you how to configure Maven to build Scala project. Fortunetly it’s very simple.<br />
<br />
To allow the compilation of Scala source code in Maven we will will require maven plugin called <span class="code">maven-scala-plugin</span> (<a href="http://scala-tools.org/mvnsites/maven-scala-plugin/">plugin home page</a>). This will take three steps:<br />
<br />
<b>First step.</b><br />
To use it we’ll have to define additional Maven Repositories in you project’s <span class="code"><project></span> tag:<br />
<pre class="brush:xml"><repositories>
<repository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>https://oss.sonatype.org/content/groups/scala-tools/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>https://oss.sonatype.org/content/groups/scala-tools/</url>
</pluginRepository>
</pluginRepositories></pre>
<br />
<i>Alternatively you can configure then in your <span class="code">settings.xml</span> file for your Maven distribution but that would make your project environment dependent.</i>
<br />
<br />
<b>Second step.</b><br />
Define a plugin for your project by adding the following section inside <span class="code"><project> -> <build> -> <plugins></span> tag:<br />
<pre class="brush:xml"><plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>${scala.maven.version}</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</pre>
<br />
<b>Third step.</b><br />
The last requirement is to define a dependency for your project containing Scala Library. To do that add this section into <span class="code"><project> -> <dependencies></span>:
<br />
<pre class="brush:xml"><dependency>
<groupid>org.scala-lang</groupid>
<artifactid>scala-library</artifactid>
<version>_here_define_scala_version_you_want_to_use_</version>
</dependency>
</pre>
<br />
<b>Result.</b><br />
This all should result in a <span class="code">pom.xml</span> that looks like this example:
<br />
<pre class="brush:xml"><?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>pl.marpiec.scala</groupId>
<artifactId>maven-example</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<scala.lang.version>2.9.1</scala.lang.version>
<scala.maven.version>2.9.1</scala.maven.version>
</properties>
<repositories>
<repository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>https://oss.sonatype.org/content/groups/scala-tools/</url>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>scala-tools.org</id>
<name>Scala-tools Maven2 Repository</name>
<url>https://oss.sonatype.org/content/groups/scala-tools/</url>
</pluginRepository>
</pluginRepositories>
<dependencies>
<!-- Scala -->
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.lang.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>${scala.maven.version}</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
</pre>
<br />
<b>Project structure.</b><br />
When you have prepared <span class="code">pom.xml</span> you can start creating your scala code. You should put it in <span class="code">src -> main -> scala</span> directory (or <span class="code">src -> test -> scala</span> for test code). This is the example from Intellij IDEA IDE:
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM1ojfTE_hxxBrZiP3IcOJbnPeG5Kwq5i5vymOLpUIiNQm-6uOogXbNBqYPPI8T5Mre4l8bs7KXISDRaJEvXYr85kdSMFe_Jf8XgaHLSztaMJFCqqqgx8GxM9Oyl1lpMXwiSe1mrQJVe0y/s1600/hello.png" imageanchor="1"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjM1ojfTE_hxxBrZiP3IcOJbnPeG5Kwq5i5vymOLpUIiNQm-6uOogXbNBqYPPI8T5Mre4l8bs7KXISDRaJEvXYr85kdSMFe_Jf8XgaHLSztaMJFCqqqgx8GxM9Oyl1lpMXwiSe1mrQJVe0y/s320/hello.png" /></a>
<br />
<br />
<b>Building.</b><br />
And you can easily build your project just by executing mvn package command, example:
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4r-xYPrBDSCq3aofJNNS1ZQTHk0Hy7AmNugGcvO958THAdN0hWdPDrbh32ftFkz-ZWA0YiozTCU_pAxPfaGCtDjzRt7m3gIUMi7xyo_lOnQ5lhffRxmFVkyHcVIOHPwkMti0DzF18iYJq/s1600/compilation.png" imageanchor="1"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4r-xYPrBDSCq3aofJNNS1ZQTHk0Hy7AmNugGcvO958THAdN0hWdPDrbh32ftFkz-ZWA0YiozTCU_pAxPfaGCtDjzRt7m3gIUMi7xyo_lOnQ5lhffRxmFVkyHcVIOHPwkMti0DzF18iYJq/s320/compilation.png" /></a>
<br />
<br />
<b>Running.</b><br />
That will create jar package that you can run by that command:<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_VVGhJxi4UIY0ZyQcaLUxJrnE0RSN8217gFKXNHuAZnhdLyHFhQQYVzl0UQL4EBgq11dZuMn6BqOAGiSL2ZJiiH80xoqU8SQndDp-kn6CKOhOJaYMjeaxWkC-HMlR6aHS-lseo8ige0rP/s1600/runhello.png" imageanchor="1"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_VVGhJxi4UIY0ZyQcaLUxJrnE0RSN8217gFKXNHuAZnhdLyHFhQQYVzl0UQL4EBgq11dZuMn6BqOAGiSL2ZJiiH80xoqU8SQndDp-kn6CKOhOJaYMjeaxWkC-HMlR6aHS-lseo8ige0rP/s320/runhello.png" /></a>
<br />
<br />
That's all, I hope that you will have a great experience using Maven and Scala for your project.Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com0tag:blogger.com,1999:blog-2508340156146834234.post-40947078738542648762013-02-28T19:00:00.000+01:002013-02-28T12:07:14.599+01:00Testing yourself in CodilityToday I've checked how <a href="http://codility.com">Codility</a> tests programmers. And it looks like a fun.<br /><br />
After you access a test (possibly to get certificate) you will have access to some simple web IDE where you will write your solution for the given problem. The task was some quite interesting algorithnic problem, to solve in 60 minutes.<br /><br />
It wasn't to hard but time limitation made it a nice challenge. I think it's quite good to test yourself and have some mental stretching.<br /><br />
Codility is used mainly as a recruitment tools for IT staff and that make me wondering how useful it is, especially to hire senior developers. Codility focuses on resolving algorithnic tasks, but in real life a developer must solve problems of much infrastructure complexity and I think that it might require different sets of skills.<br /><br />
But obviously it's nice tools to verify some basic programming skills of possible junior developers.<br />
And in the end, this is my Certificate gained in Codility:<br />
<iframe src="http://www.codility.com//cert/badge/certK749K2-HAFV4NG3Y9WS3K2Y/" width="125" height="75" style="border:0;" >
<a href="http://www.codility.com//cert/view/certK749K2-HAFV4NG3Y9WS3K2Y/">Codility badge</a>
</iframe>Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com4tag:blogger.com,1999:blog-2508340156146834234.post-38069031798412687182012-04-16T18:19:00.000+02:002012-04-16T18:48:40.684+02:00Scala, gson and OptionDuring development of my pet project written in Scala I've run into a problem when trying to serialize Option object into Json using gson library. This is problem, because Gson have implemented serializers for many Java objects and primitives but it can't handle Option object properly.<br />
<br />
<b>Solution</b> <br />
<br />
The solution is to impelment your own OptionSerializer and register it during creation of Gson instance:<br />
<br />
<pre class="brush:scala">
class OptionSerializer extends JsonSerializer[Option[Any]] with JsonDeserializer[Option[Any]] {
def serialize(src: Option[Any], typeOfSrc: Type, context: JsonSerializationContext): JsonElement
def deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext): Option[Any]
}
</pre>
<pre class="brush:scala">val gsonBuilder = new GsonBuilder
gsonBuilder.registerTypeAdapter(classOf[Option[Any]], new OptionSerializer)
val gson = gsonBuilder.create
</pre>
<br />
Lets start with implementing serialize method.<br />
<br />
First will check if given src is None value. If it is None, then serialize method will return an empty JsonObject. It is important to use empty JsonObject instead of JsonNull, because during deserialization gson will skip value if it is null. And we would like it to set the None value, so we wouldn't end with null.<br />
<br />
If the Option is not empty we need to serialize its content using context.serialize method. So our serialize method should look like that:<br />
<pre class="brush:scala">def serialize(src: Option[Any], typeOfSrc: Type, context: JsonSerializationContext): JsonElement = {
if (src.isDefined) {
context.serialize(src.get.value)
} else {
new JsonObject
}
}
</pre>
<br />
There is a <b>problem </b>with interoperability between Java and Scala, when using parametrized types (generics) with primitives. Simply it is impossible in Java (so the code Option<int> won't compile), but it is legal to write Option[Int] in Scala.<br />
<br />
That results in a problem that Option of simple types (Int, Long, Boolean etc.) looses information about underlying type during runtime (passed in ParameterizedType), causing that we will have Option[Object]. And we need to know type of object contained by Option during deserialization.<br />
<br />
The solution is to save type information inside json. It isn't very elegant solution, but it works very well.<br />
<br />
So the final serialize method will look like that:<br />
</int><br />
<pre class="brush:scala">def serialize(src: Option[Any], typeOfSrc: Type, context: JsonSerializationContext): JsonElement = {
val jsonObject = new JsonObject
if (src.isDefined) {
def value = src.get
jsonObject.addProperty("class", value.asInstanceOf[Object].getClass.getName)
jsonObject.add("value", context.serialize(value))
}
jsonObject
}
</pre>
<br />
When the problems during serialization are resolved, it is quite easy to write deserialization method. We only need to check if the given jsonElement is not empty, and in that case return None. Otherwise we create Option object containing deserialized value.<br />
<br />
<pre class="brush:scala">def deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext):Option[Any] = {
if (json.isJsonNull) {
None
} else if (json.isJsonObject && json.getAsJsonObject.entrySet().size()==0) {
None
} else {
val className = json.getAsJsonObject.get("class").getAsString
val deserialized = context.deserialize(json.getAsJsonObject.get("value"), Class.forName(className))
Option(deserialized )
}
}
</pre>
<br />
Below is the example of json created from simple object:<br />
<br />
Class:<br />
<pre class="brush:scala">class OptionalDataObject(var intOption:Option[Int], stringOption:Option[String])
</pre>
<br />
Json:<br />
<pre class="brush:javascript">{
"intOption":{"class":"java.lang.Integer","value":3},
"stringOption":{"class":"java.lang.String","value":"test"}
}
</pre>
<br />
<b>Result:</b><br />
And in the end the whole OptionSerializer class:<br />
<br />
<pre class="brush:scala">class OptionSerializer extends JsonSerializer[Option[Any]] with JsonDeserializer[Option[Any]] {
def serialize(src: Option[Any], typeOfSrc: Type, context: JsonSerializationContext): JsonElement = {
val jsonObject = new JsonObject
if (src.isDefined) {
def value = src.get
jsonObject.addProperty("class", value.asInstanceOf[Object].getClass.getName)
jsonObject.add("value", context.serialize(value))
}
jsonObject
}
def deserialize(json: JsonElement, typeOfT: Type, context: JsonDeserializationContext):Option[Any] = {
if (json.isJsonNull) {
None
} else if (json.isJsonObject && json.getAsJsonObject.entrySet().size()==0) {
None
} else {
val className = json.getAsJsonObject.get("class").getAsString
val deserialized = context.deserialize(json.getAsJsonObject.get("value"), Class.forName(className))
Option(deserialized )
}
}
}
</pre>Marcin Pieciukiewiczhttp://www.blogger.com/profile/14703396936344720814noreply@blogger.com1