Scala provides a data structure, the array, which stores a fixed-size sequential collection of elements of the equivalent type. An array is used to store a collection of knowledge, but it’s often more useful to think of an array as a collection of variables of the equivalent type.

Instead of declaring individual variables, such as number0, number1, …, and number99, you declare one array variable such as numbers and use numbers[0], numbers[1], and …, numbers[99] to represent individual variables. This tutorial introduces how to declare array variables, create arrays, and process arrays using indexed variables. The index of the first element of an array is the number zero and the index of the last element is the total number of elements minus one.

Declaring Array Variables

To use an array in a program, you must declare a variable to reference the array and you must specify the type of array the variable can reference.

The subsequent is the syntax for declaring an array variable.

Syntax

var z:Array[String] = new Array[String](3)




or




var z = new Array[String](3)

Here, z is declared as an array of Strings that may hold up to three elements. Values can be assigned to individual elements or get access to individual elements, it can be done by using commands like the subsequent −

Command

z(0) = "Zara"; z(1) = "Nuha"; z(4/2) = "Ayan"

Here, the last example shows that in general the index can be any expression that yields a whole number. There is one more way of defining an array −

var z = Array("Zara", "Nuha", "Ayan")

Subsequent picture represents an array myList. Here, myList holds ten double values and the indices are from 0 to 9.

Processing Arrays

When processing array elements, we often use loop contol structures because all of the elements in an array are of the equivalent type and the size of the array is known.

Below is an example program of showing how to create, initialize and process arrays −

Example

object Demo {

   def main(args: Array[String]) {

      var myList = Array(1.9, 2.9, 3.4, 3.5)

     

      // Print all the array elements

      for ( x <- myList ) {

         println( x )

      }




      // Summing all elements

      var total = 0.0;

     

      for ( i <- 0 to (myList.length - 1)) {

         total += myList(i);

      }

      println("Total is " + total);




      // Finding the largest element

      var max = myList(0);

     

      for ( i <- 1 to (myList.length - 1) ) {

         if (myList(i) > max) max = myList(i);

      }

     

      println("Max is " + max);

   }

}

Save the above program in Demo.scala. The subsequent commands are used to compile and execute this program.

Command

\>scalac Demo.scala

\>scala Demo

Output

1.9

2.9

3.4

3.5

Total is 11.7

Max is 3.5

Scala does not directly support various array operations and provides various methods to process arrays in any dimension. If you want to use the different methods then it’s required to import Array._ package.

Multi-Dimensional Arrays

There are many situations where you would need to define and use multi-dimensional arrays (i.e., arrays whose elements are arrays). For example, matrices and tables are examples of structures that can be realized as two-dimensional arrays.

The subsequent is the example of defining a two-dimensional array −

var myMatrix = ofDim[Int](3,3)

This is an array that has three elements each being an array of integers that has three elements.

Try the subsequent example program to process a multi-dimensional array −

Example

import Array._




object Demo {

   def main(args: Array[String]) {

      var myMatrix = ofDim[Int](3,3)

     

      // build a matrix

      for (i <- 0 to 2) {

         for ( j <- 0 to 2) {

            myMatrix(i)(j) = j;

         }

      }

     

      // Print two dimensional array

      for (i <- 0 to 2) {

         for ( j <- 0 to 2) {

            print(" " + myMatrix(i)(j));

         }

         println();

      }

   }

}

Save the above program in Demo.scala. The subsequent commands are used to compile and execute this program.

Command

\>scalac Demo.scala

\>scala Demo

Output

0 1 2

Concatenate Arrays

Try the subsequent example which makes use of concat() method to concatenate two arrays. You can pass more than one array as arguments to concat() method.

Example

import Array._




object Demo {

   def main(args: Array[String]) {

      var myList1 = Array(1.9, 2.9, 3.4, 3.5)

      var myList2 = Array(8.9, 7.9, 0.4, 1.5)




      var myList3 =  concat( myList1, myList2)

     

      // Print all the array elements

      for ( x <- myList3 ) {

         println( x )

      }

   }

}

Save the above program in Demo.scala. The subsequent commands are used to compile and execute this program.

Command

\>scalac Demo.scala

\>scala Demo

Output

1.9

2.9

3.4

3.5

8.9

7.9

0.4

1.5

Create Array with Range

Use of range() method to generate an array containing a sequence of increasing integers in a given range. You can use final argument as step to create the sequence; if you do not use final argument, then step would be assumed as 1.

Let us take an example of creating an array of range (10, 20, 2): It means creating an array with elements between 10 and 20 and range difference 2. Elements in the array are 10, 12, 14, 16, and 18.

Another example: range (10, 20). Here range difference is not given so by default it assumes 1 element. It create an array with the elements in between 10 and 20 with range difference 1. Elements in the array are 10, 11, 12, 13, …, and 19.

The subsequent example program shows how to create an array with ranges.

Example

import Array._




object Demo {

   def main(args: Array[String]) {

      var myList1 = range(10, 20, 2)

      var myList2 = range(10,20)




      // Print all the array elements

      for ( x <- myList1 ) {

         print( " " + x )

      }

     

      println()

      for ( x <- myList2 ) {

         print( " " + x )

      }

   }

}

Save the above program in Demo.scala. The subsequent commands are used to compile and execute this program.

Command

\>scalac Demo.scala

\>scala Demo

Output

10 12 14 16 18

10 11 12 13 14 15 16 17 18 19

Scala Array Methods

Subsequent are the important methods, which you can use while playing with array. As shown above, you would have to import Array._ package before using any of the mentioned methods. For a complete list of methods available, please check official documentation of Scala.

Sr.No	Methods with Description
1	*def apply( x: T, xs: T ): Array[T]** Creates an array of T objects, where T can be Unit, Double, Float, Long, Int, Char, Short, Byte, Boolean.
2	*def concat[T]( xss: Array[T] ): Array[T]** Concatenates all arrays into a single array.
3	def copy( src: AnyRef, srcPos: Int, dest: AnyRef, destPos: Int, length: Int ): Unit Copy one array to another. Equivalent to Java’s System.arraycopy(src, srcPos, dest, destPos, length).
4	def empty[T]: Array[T] Returns an array of length 0
5	def iterate[T]( start: T, len: Int )( f: (T) => T ): Array[T] Returns an array containing repeated applications of a function to a start value.
6	def fill[T]( n: Int )(elem: => T): Array[T] Returns an array that contains the leads of some element computation a number of times.
7	def fill[T]( n1: Int, n2: Int )( elem: => T ): Array[Array[T]] Returns a two-dimensional array that contains the leads of some element computation a number of times.
8	def iterate[T]( start: T, len: Int)( f: (T) => T ): Array[T] Returns an array containing repeated applications of a function to a start value.
9	def ofDim[T]( n1: Int ): Array[T] Creates array with given dimensions.
10	def ofDim[T]( n1: Int, n2: Int ): Array[Array[T]] Creates a 2-dimensional array
11	def ofDim[T]( n1: Int, n2: Int, n3: Int ): Array[Array[Array[T]]] Creates a 3-dimensional array
12	def range( start: Int, end: Int, step: Int ): Array[Int] Returns an array containing equally spaced values in some integer interval.
13	def range( start: Int, end: Int ): Array[Int] Returns an array containing a sequence of increasing integers in a range.
14	def tabulate[T]( n: Int )(f: (Int)=> T): Array[T] Returns an array containing values of a given function over a range of integer values starting from 0.
15	def tabulate[T]( n1: Int, n2: Int )( f: (Int, Int ) => T): Array[Array[T]] Returns a two-dimensional array containing values of a given function over ranges of integer values starting from 0.

So, this brings us to the end of blog. This Tecklearn ‘How to use Arrays in Scala Programming Language’ helps you with commonly asked questions if you are looking out for a job in Apache Spark and Scala and Big Data Developer. If you wish to learn Apache Spark and Scala and build a career in Big Data Hadoop domain, then check out our interactive, Apache Spark and Scala Training, that comes with 24*7 support to guide you throughout your learning period. Please find the link for course details:

https://www.tecklearn.com/course/apache-spark-and-scala-certification/

Apache Spark and Scala Training

About the Course

Tecklearn Spark training lets you master real-time data processing using Spark streaming, Spark SQL, Spark RDD and Spark Machine Learning libraries (Spark MLlib). This Spark certification training helps you master the essential skills of the Apache Spark open-source framework and Scala programming language, including Spark Streaming, Spark SQL, machine learning programming, GraphX programming, and Shell Scripting Spark. You will also understand the role of Spark in overcoming the limitations of MapReduce. Upon completion of this online training, you will hold a solid understanding and hands-on experience with Apache Spark.

Why Should you take Apache Spark and Scala Training?

The average salary for Apache Spark developer ranges from approximately $93,486 per year for Developer to $128,313 per year for Data Engineer. – Indeed.com
Wells Fargo, Microsoft, Capital One, Apple, JPMorgan Chase & many other MNC’s worldwide use Apache Spark across industries.
Global Spark market revenue will grow to $4.2 billion by 2022 with a CAGR of 67% Marketanalysis.com

What you will Learn in this Course?

Introduction to Scala for Apache Spark

What is Scala
Why Scala for Spark
Scala in other Frameworks
Scala REPL
Basic Scala Operations
Variable Types in Scala
Control Structures in Scala
Loop, Functions and Procedures
Collections in Scala
Array Buffer, Map, Tuples, Lists

Functional Programming and OOPs Concepts in Scala

Functional Programming
Higher Order Functions
Anonymous Functions
Class in Scala
Getters and Setters
Custom Getters and Setters
Constructors in Scala
Singletons
Extending a Class using Method Overriding

Introduction to Spark

Introduction to Spark
How Spark overcomes the drawbacks of MapReduce
Concept of In Memory MapReduce
Interactive operations on MapReduce
Understanding Spark Stack
HDFS Revision and Spark Hadoop YARN
Overview of Spark and Why it is better than Hadoop
Deployment of Spark without Hadoop
Cloudera distribution and Spark history server

Basics of Spark

Spark Installation guide
Spark configuration and memory management
Driver Memory Versus Executor Memory
Working with Spark Shell
Resilient distributed datasets (RDD)
Functional programming in Spark and Understanding Architecture of Spark

Playing with Spark RDDs

Challenges in Existing Computing Methods
Probable Solution and How RDD Solves the Problem
What is RDD, It’s Operations, Transformations & Actions Data Loading and Saving Through RDDs
Key-Value Pair RDDs
Other Pair RDDs and Two Pair RDDs
RDD Lineage
RDD Persistence
Using RDD Concepts Write a Wordcount Program
Concept of RDD Partitioning and How It Helps Achieve Parallelization
Passing Functions to Spark

Writing and Deploying Spark Applications

Creating a Spark application using Scala or Java
Deploying a Spark application
Scala built application
Creating application using SBT
Deploying application using Maven
Web user interface of Spark application
A real-world example of Spark and configuring of Spark

Parallel Processing

Concept of Spark parallel processing
Overview of Spark partitions
File Based partitioning of RDDs
Concept of HDFS and data locality
Technique of parallel operations
Comparing coalesce and Repartition and RDD actions

Machine Learning using Spark MLlib

Why Machine Learning
What is Machine Learning
Applications of Machine Learning
Face Detection: USE CASE
Machine Learning Techniques
Introduction to MLlib
Features of MLlib and MLlib Tools
Various ML algorithms supported by MLlib

Integrating Apache Flume and Apache Kafka

Why Kafka, what is Kafka and Kafka architecture
Kafka workflow and Configuring Kafka cluster
Basic operations and Kafka monitoring tools
Integrating Apache Flume and Apache Kafka

Apache Spark Streaming

Why Streaming is Necessary
What is Spark Streaming
Spark Streaming Features
Spark Streaming Workflow
Streaming Context and DStreams
Transformations on DStreams
Describe Windowed Operators and Why it is Useful
Important Windowed Operators
Slice, Window and ReduceByWindow Operators
Stateful Operators

Improving Spark Performance

Learning about accumulators
The common performance issues and troubleshooting the performance problems

DataFrames and Spark SQL

Need for Spark SQL
What is Spark SQL
Spark SQL Architecture
SQL Context in Spark SQL
User Defined Functions
Data Frames and Datasets
Interoperating with RDDs
JSON and Parquet File Formats
Loading Data through Different Sources

Scheduling and Partitioning in Apache Spark

Concept of Scheduling and Partitioning in Spark
Hash partition and range partition
Scheduling applications
Static partitioning and dynamic sharing
Concept of Fair scheduling
Map partition with index and Zip
High Availability
Single-node Recovery with Local File System and High Order Functions

Got a question for us? Please mention it in the comments section and we will get back to you.

373