Friday, February 9, 2018

Overview of Scala

  • Is a general-purpose programming language providing support for functional programming and a strong static type system. 
  • Designed to be concise, many of Scala's design decisions aimed to address criticisms of Java
  • Source code is intended to be compiled to Java bytecode, so that the resulting executable code runs on a Java virtual machine. Scala provides language interoperability with Java
  • Scala is object-oriented, and  has many features of functional programming languages like Scheme, Standard ML and Haskell, including currying, type inference, immutability, lazy evaluation, and pattern matching

Scala programming with Spark

Spark is installed in cloudera installation

Spark Shell
Command: cd /usr/lib/spark


Mutable and immutable variables
Difference between var and val. we can reassign a var variable, however val are immutable.They cannot be reassigned.
scala> val msg="How is your work"
msg: String = How is your work

scala> msg="How id your work"
<console>:27: error: reassignment to val
         msg="How id your work"

scala> var msg="I am mutable"
msg: String = I am mutable

scala> msg="I did prove that I am Immutable"
msg: String = I did prove that I am Immutable


Type Inference
It is often not necessary in Scala to specify the type of a variable, since the compiler can deduce the type from the initialization expression of the variable.Once the type is assigned. It remains same for the entire scope

In the example below we are trying to reassign a int value to a string variable .
scala> msg="I did prove that I am Immutable"
msg: String = I did prove that I am Immutable

scala> msg=10
<console>:27: error: type mismatch;
 found   : Int(10)
 required: String

{} is the list of expressions and result is the list of expressions.The value of the block is the value of the last expression of it
scala> val x={val a=20;val b=100;b-a}
x: Int = 80

Lazy values We can define a variable as lazy.Initialization is deferred, till it is accessed for the first time. They are very useful for delaying costly initialization instructions. Lazy values do not give an error.

scala>  lazy val"UnitesStates").mkString
file: String = <lazy>

If Else
  • In scala if-else has a value, of the expression following it semicolons are optional in scala Every expression in scala has a type.
  • First if a statement has a type Int second statement has a type any. 
  • Type of mixed expression is super type of both branches
scala> var x=10
x: Int = 10

scala> val s=if(x>0 && x<6)1 else 0
s: Int = 0

While Loop:
In scala while and do-while loops are same as java

scala> var value=4 
value: Int = 4

scala> var i=0
i: Int = 0

scala> while(i<=value)
     | {
     | println(i)
     | i+=1
     | }
The ++i or i++ are don’t work in scala.
You will have to use i+=1 or i=i+1 expressions instead

For Each Loop

scala> var args="hi"
args: String = hi

scala> args.foreach(arg=>println(arg))

For loop Scala doesn’t have for (initialize; test; update) syntax
scala> for (i<-1 to 5)
     | println(i)

Traversing an array, following could be applied
scala> val str="Hello world"
str: String = Hello world

scala> var sum=0
sum: Int = 0

scala> for(i<-0 until str.length)sum+=i

scala> print(sum)

Advanced for loop:
we can have multiple generators in for loop
scala> print(sum)
scala> for(i<-1 to 3;j<-1 to 3) println(10*i+j)

We can put conditions in multi generators for loop.
scala> for(i<-1 to 3;j<-1 to 3 if i==j)println(10*i+j)

We can introduce variables in loop!
scala> for(i<-1 to 3;x=4-i;j<-x to 3)println(10*i+j)

If the body of for loop starts with yield, it returns a collection of values. var x=for(i<-1 --="" 10.0="" 12.5="" 15.0="" 17.5="" 20.0="" 20="" 22.5="" 25.0="" 27.5="" 30.0="" 32.5="" 35.0="" 37.5="" 40.0="" 42.5="" 45.0="" 47.5="" 5.0="" 50.0="" 7.5="" always="" and="" are="" both="" but="" concept="" functions="" generated="" has="""" html="" i="" in="" invoked="" is="" java="" methods.="" methods="" not.="" objects="" on="" ouble="" scala.collection.immutable.indexedseq="" scala="" similar="" static="" support="" this="" to="" using="" vector="" x:="" yield="">
scala> def area(radius:Double):Double={3.14*radius*radius}
area: (radius: Double)Double

scala> area(10.09)
res7: Double = 319.677434

scala> def area(radius:Int):Double={3.14*radius*radius}
area: (radius: Int)Double

scala> area(10)
res8: Double = 314.0


  • Scala has support both methods and functions 
  • Methods are always invoked on objects, but functions are not. In java this concept is similar to static methods. 
  • We do not need return statement

scala> def area(radius:Double):Double={3.14*radius*radius}
area: (radius: Double)Double

scala> area(10.09)
res7: Double = 319.677434

scala> def area(radius:Int):Double={3.14*radius*radius}
area: (radius: Int)Double

scala> area(10)
res8: Double = 314.0

Recursive functions 
We need to specify a data type for a recursive function
def factorial(n:Int):Int=if(n==0)1 else n*factorial(n-1)
factorial: (n: Int)Int

scala> factorial(5)
res9: Int = 120


  • Scala provides a data structure, the array, which stores a fixed-size sequential collection of elements of the same type. 
  • An array is used to store a collection of data, but it is often more useful to think of an array as a collection of variables of the same type

scala> val st=Array("Hello", "World")
st: Array[String] = Array(Hello, World)

scala> val n=new Array[Int](10)
n: Array[Int] = Array(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)

scala> val n=new Array[String](10)
n: Array[String] = Array(null, null, null, null, null, null, null, null, null, null)

scala> val st=Array("Hello", "World")
st: Array[String] = Array(Hello, World)

Variable Length Arrays (Array buffers). Similar to java ArrayList
scala> a+=1
res11: a.type = ArrayBuffer(1)

scala> a+=(2,3,5)
res12: a.type = ArrayBuffer(1, 2, 3, 5)

scala> a++=Array(6,7,8)
res13: a.type = ArrayBuffer(1, 2, 3, 5, 6, 7, 8)

Arrays and ArrayBuffers:
sCommon Operations
a.trimEnd(3) //remove last 3 elements
a.insert(2,9) //adds element at 2nd Index
a.insert(2,10,11,12) //adds a list
a.remove(3) //removes an element
a.remove(2,3) //removes three elements from index 2

More functions on Array
scala> Array(1,2,3,4).sum
res22: Int = 10

scala> Array(1,5,9,8).max
res23: Int = 9

scala> val a=Array(1,7,2,9)
a: Array[Int] = Array(1, 7, 2, 9)

scala> scala.util.Sorting.quickSort(a)

scala> a.mkString("**")
res25: String = 1**2**7**9

Traversing and transformation:
scala> for(el<-a)
     | println(el)

scala> for(el<-a if el%2==0)yield(2*el)
res28: Array[Int] = Array(4)

 In scala, Map in a collection of pair. A pair is a group of two values (Not necessarily of same type).
scala> val mapping=Map("UnitedStates"-> "UnitedKingdom",
mapping: scala.collection.immutable.Map[String,String] = Map(UnitedStates -> 
UnitedKingdom, Tokyo -> India)

scala> val mapping=scala.collection.mutable.
Map("UnitedStates"-> "U","Tokyo"->"I")
mapping: scala.collection.mutable.Map[String,String] = Map(Tokyo -> I, UnitedStates -> U)

scala> val x=mapping("UnitedStates")
x: String = U

scala> val x=mapping.getOrElse("Unit",0)
x: Any = 0

Accessing Maps
scala> mapping -="UnitedStates"
res0: mapping.type = Map(Tokyo -> I)

scala> mapping +=("Irland6"-> "Newzealand")
res1: mapping.type = Map(Tokyo -> I, Irland6 -> Newzealand)

scala> for((u,i)<-mapping)yield(i,u)
res2: scala.collection.mutable.Map[String,String] = Map(I -> Tokyo, Newzealand -> Irland6)

Tuple is more generalized form of pair. Tuple has more than two values of potentially different types
scala> val a=(1,4,"Bob","Jack")
a: (Int, Int, String, String) = (1,4,Bob,Jack)

scala> a._2
res0: Int = 4

scala> a _2
warning: there were 1 feature warning(s); re-run with -feature for details
res1: Int = 4

In tuples the offset start with 1 and NOT from 0. Tuples are typically used for the functions which return more than one value.
scala> "New jercy".partition(_.isUpper)
res2: (String, String) = (N,ew jercy)

Scala Lists are quite similar to arrays which means, all the elements of a list have the same type but there are two important differences.
First, lists are immutable, which means elements of a list cannot be changed by assignment. Second, lists represent a linked list whereas arrays are flat.
scala> val lst=List(1,2)
lst: List[Int] = List(1, 2)

scala> lst.head
res3: Int = 1

scala> lst.tail
res4: List[Int] = List(2)

Operator adds a new list from given head and tail.
scala> 2::List(4,5)
res5: List[Int] = List(2, 4, 5)

We can use iterator to iterate over a list, but recursion is a preferred practice in scala
def sum(l:List[Int]):Int={if(l==Nil)0 else l.head + sum(l.tail)}
sum: (l: List[Int])Int

scala> val y=sum(lst)
y: Int = 3

No comments:

Post a Comment